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PREFACE 


The  growth  of  the  electronics  technology  has  created  challenges  to  the  use  of 
electronics  systems  in  several  respects.  Not  the  least  of  these  challenges  is 
represented  by  the  complexity  of  testing  the  systems  in  the  field  to  determine 
functional  status  and  to  permit  efficient  fault  detection  and  fault  isolation. 

The  Rome  Air  Development  Center  (RADC)  through  its  Testability  Research  Pro¬ 
gram  has  provided  Government  and  Industry  with  Improved  testability  design 
tools  and  software.  The  Testability  Notebook  is  an  RADC-sponsored  effort  to 
define  and  synopsize  a  systematic  methodology  for  Including  a  high  degree  of 
testability  in  the  evolving  development  of  electronics  systems. 

The  RADC  Testability  Notebook  consists  of  an  ordered  and  related  collection  of 
tasks  oriented  towards  the  evolution  of  prime  system  testability  from  concept 
formulation  through  deployed  operations.  The  primary  objective  is  to  provide 
for  achievement  of  cost-effective  and  efficient  field  testability  of  electron¬ 
ic  systems. 

It  is  hoped  that  this  Notebook  will  be  the  first  milestone  In  the  rapid 
implementation  of  a  simplified  methodology  for  achieving  the  desired  levels  of 
testability  and  that  high  visibility  mqy  be  accorded  to  the  resultant  benefits 
of  higher  operational  readiness  at  reduced  cost. 

Mr.  James  Saporito  was  the  RADC  Project  Engineer  on  this  effort  from  its 
inception  until  his  retirement.  Requests  for  Information  concerning  current 
and  future  additions  to  the  Testability  Notebook  and  other  comments  should  be 
directed  to: 

Jerome  Kllon 
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Grlfflss  AFB  NY  13441 
AC  315-330-4726 
AV:  587-4726 
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1.1  PURPOSE 


RADC  TESTABILITY  NOTEBOOK 
INTRODUCTION 


The  RADC  Testability  Notebook  presents  a  consolidation  of  information 
presently  availably  relating  to  testability  design  techniques,  procedures, 
cost  tradeoff  tools,  and  the  relationship  of  testability  to  other  design 
disciplines  and  requirements. 

1.2  STATEMENT  OF  PROBLEM 

In  the  past  testability  design  considerations  have  comprised  weak 
links  in  the  chain  from  system/equipment  specification,  through  design,  to 
the  definition  of  built-in-test  (BIT)  and  external  test  equipment  (ETE) 
support  hardware  and  software.  Testability  of  the  prime  equipment  has 
generally  become  a  concern  only  during  the  AGE  or  GSE  design  effort  which 
normally  has  occurred  late  in  the  development  phase,  when  the  operational 
hardware  design  is  firm;  changes  to  improve  testability  are  thereupon 
assigned  low  priority  because  of  associated  schedule  delays  and  higher 
costs.  The  resultant  effect  is  interaction  into  operational  use  of  elec¬ 
tronic  equipments  and  systems,  which  have  high  mean  times  to  repair,  high 
manpower  rates  (maintenance  manhours  per  operating  hour),  retest  OK 
(RETOK),  could  not  duplicate  (CND),  and  false  alarm  rates.  These,  in  turn, 
reduce  system  availability  and  operational  readiness  and  Increase  life 
cycle  costs. 

System/Equipment  testability  can  be  improved  by  treating  testability 
as  a  design  discipline  starting  In  the  conceptual  phase  and  continuing 
through  the  acquisition  process.  Such  a  discipline  when  applied  by  manage¬ 
ment  elevates  testability  to  design  factor  stature  as  a  peer  to  reliabil¬ 
ity,  maintainability,  availability  and  supportability.  The  nature,  makeup 
and  organization  of  that  discipline,  as  It  should  be  applied  to  each  phase 
of  the  equipment/system  acquisition  process  is  contained  In  this  Notebook. 

1.3  Definition  of  Testability  Functions.  Tasks,  and  Interfaces 

The  Notebook  takes  cognizance  of  established  system  development  phas¬ 
ing  as  the  baseline  or  driving  set  of  events  for  Its  structure.  The  five 
program  phases  were  taken  as  Conceptual,  Validation,  Full  Scale  Develop¬ 
ment,  Production  and  Deployment/Operation.  System  development  objectives 
for  each  phase  were  analyzed  and  corresponding  testability  postures  and 
objectives  were  then  established  for  each  phase,  and  a  set  of  testability 
functions  and  tasks  were  derived. 

The  Notebook  In  addition  treats  two  broad  Interface  areas  within  a 
testability  program.  First,  the  set  of  relationships  among  the  testabil¬ 
ity  tasks  within  a  program  phase  and  from  one  phase  to  another.  The  second 
Is  the  Interface  area  between  testability  and  Its  related  disciplines 
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including  system  and  unit  design,  developmental  test,  maintainability, 
reliability,  logistics  and  support ability.  Program  management  makeup  and 
guidance  are  provided  for  these  as  well  as  for  the  primary  testability 
activities  and  tasks. 

1.4  TECHNICAL  RESULTS 

The  results  of  the  Testability  Notebook  are  documented  in  the  detailed 
testability  program  flow  charts,  testability  task  compendia,  and,  appllca- 
tion  notes  which  forms  Its  body. 

1.5  Testability  Program  Flow 

The  flow  charts  indicate  the  functions,  tasks,  and  interfaces,  their 
order  and  logic  applicable  to  each  program  phase.  The  flow  charts  in 
addition  indicate  the  prerequisite  activities  necessary  for  the  accomp¬ 
lishment  of  any  set  of  functions  and  tasks.  Figure  2-1  (page  2-14)  pre¬ 
sents  the  baseline  overall  task  flow.  Each  phase  is  broken  Into  functional 
activity  areas,  each  of  which  encloses  the  tasks  of  that  activity.  A 
separate  section  of  the  compendia  treats  the  tasks  of  each  phase  and 
includes  a  flow  chart  for  the  phase. 

1.6.  Testability  Task  Compendia 

Each  flow  chart  references  a  different  set  of  testability  task  com¬ 
pendia  which  are  coded  by  phase  and  function.  The  compendia  In  the  Note¬ 
book  are  organized  with  respect  to  program  phase  and  each  phase  is  broken 
down  into  its  relevant  testability  functions.  Each  function  is  then  furth¬ 
er  broken  down  into  its  constituent  tasks  and  activities.  Technical  guid¬ 
ance  is  then  provided  for  each  task/activity. 

1.6.1  Testability  Functions 

The  baseline  set  of  testability  functions  is  summarized  as  follows. 
Each  function  Is  developed  into  one  or  more  tasks  and  presented  In  the 
Testability  Notebook. 

Conceptual  Phase 

o  Establish  the  qualitative  and  quantitative  testability  require¬ 
ments  for  the  prime  system. 

o  Conduct  preliminary  tradeoffs  to  establish  the  test  system  defini¬ 
tion  and  to  provide  design  criteria  for  prime  system  compatibility 
with  the  test  philosophy. 

o  Incorporate  testability  requirements  Into  the  system  specifica¬ 
tion. 

Validation  Phase 

o  Provide  a  Testability  Program  Plan. 
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o  Provide  detailed  guidance  for  Incorporation  of  testability  In  the 
prime  system  design. 

o  Perform  detailed  tradeoffs  of  Built-in-Test  and  external  test 
equipment  mixes. 

o  Analyze  the  testability  characteristics  of  the  evolving  design, 
o  Document  the  results  of  the  testability  analysis, 
o  Develop  specification  segments  for  test  and  testability  elements, 
o  Support  and  participate  In  the  Preliminary  Design  Review. 

Full  Scale  Development  Phase 
o  Analyze  and  document  detailed  test  requirements, 
o  Predict  effectiveness  of  the  defined  test  system, 
o  Document  testability  costs  and  benefits, 
o  Support  and  participate  In  the  Critical  Design  Review, 
o  Plan  and  conduct  a  Testability  Demonstration, 
o  Document  a  final  testability  analysis, 
o  Monitor  developmental  test. 

o  Conduct  a  pre-production  testability  readiness  review. 

Production  Phase 

o  Monitor  production  and  participate  In  change  proposal  activity. 

Operations/Support  Phase 
o  Monitor  and  evaluate  field  testability  data. 


2.0  TESTABILITY  NOTEBOOK  -  SUMMARY 


2.1  Technical  Description  of  the  Testability  Problem 

Testability  is  a  significant  key  to  achieving  system  performance  and  cost- 
effectiveness  goals.  A  systematic  approach  is  needed  to  establish  and  meet 
testability  requirements  beginning  in  the  earliest  program  phases. 

2.1.1  The  Requirement  for  a  Disciplined  Approach  to  Testability  Design 

Increasing  recognition  is  being  given  to  correlations  between  system 
life  cycle  costs  and  the  systems'  testability  characteristics.  Reliability, 
maintainability,  availability  and  producibility,  among  other  disciplines,  are 
essentially  peers  of  testability,  yet  in  modern  program  developments  they  are 
frequently  treated  much  more  formally  than  testability.  Steps  must  therefore 
be  taken  to  elevate  testability  to  a  higher  status,  with  the  objective  of 
realizing  significant  long  term  cost  benefits. 

2. 1.1.1  Testability  Definition:  Testability  may  be  regarded  as  the  inherent 
ability  of  an  item  to  undergo  valid,  dependable  functional  testing  and  associ¬ 
ated  fault  detection/isolation,  within  constraints  of  elapsed  time,  complexity 
of  access,  support  equipment  and  functional  procedures,  and  within  set  limits 
of  manpower,  material,  and  other  resources.  The  tested  item  may  be  any  level  of 
indenture  of  a  system  of  prime  mission  equipment  or  of  some  tangible  element  of 
mission  support.  Table  2-1  expresses  this  concept. 

2. 1.1. 2  Testability  Objectives;  Functional  test  and  condition  monitoring  are 
necessary  to  give  assurance  and  expectation  of  mission  success  preparatory  to 
or  during  operation,  and  in  the  course  of  maintenance  or  repair.  Malfunction 
detection  Is  necessary  to  permit  consideration  of  alternative  modes  of  opera¬ 
tion  and  degree  of  mission  success  to  be  expected  from  use  of  each  alternative 
mode.  Annunciation  of  the  malfunction  is  a  prerequisite  to  making  decisions  to 
conduct  maintenance  and  aids  in  determining  whether  or  not  maintenance  will 
take  place  with  or  without  system  shutdown.  Isolation  of  malfunctions  is  in 
turn  a  prerequisite  to  effecting  repairs  or  otherwise  restoring  degraded  com¬ 
ponents  to  required  levels  of  operating  performance. 

2. 1.1. 3  Testability  Approaches:  Table  2-2  summarizes  testability  approaches. 
Testability  characteristics  must  be  injected  early  into  designs.  Poor  test¬ 
ability  conditions  constitute  design  flaws  and  will  lead  to  expensive  design 
change  procedures  if  not  recognized  before  the  design  baseline  is  established. 
This  need  for  identification  and  resolution  applies  from  the  inception  of 
conceptual  studies,  through  validation,  engineering  development,  full  scale 
development,  production  and  the  deployment  phases.  Testability  is  subordinate 
to  performance,  but  system  performance,  to  be  effective,  efficient,  and  eco¬ 
nomical,  requires  testability  in  design. 
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2. 1.1. 4  Testability  in  The  System  Heirarchy:  Faults  may  be  apparent  at  any  point  in 
the  system  heirarchy  beginning  with  the  initial  fabrication  or  forming  of  single 
components .  Testability  therefore  needs  to  be  considered  at  the  various  stages  of 
fabrication  and  assembly,  and  at  the  various  steps  of  integrating  assemblies  into 
LRUs,  LRU's  into  units,  units  into  equipments,  and  equipment  into  systems.  Further* 
more,  testability  is  a  feature  needed  to  ease  and  simplify  acceptance  and  demonstra* 
tion,  to  monitor  deployed  performance,  as  well  as  to  detect,  locate  and  isolate 
faults. 

2. 1.1. 5  Testability  Economics  and  Impacts.  Properly  managed,  effective  testability 
in  design  may  be  achieved  without  adding  significant  costs.  Lack  of  effective 
testability  incurs  expense  of  much  greater  magnitude  than  the  cost  of  achieving 
testability  in  design.  Observe  the  MTTR  term  in  the  classical  formulation  of 
inherent  availability, 

Aj  *  (MTBF) / (MTBF  +  MTTR). 

Poor  testability  characteristics  first  of  all  cause  extended  MTTR,  with  the  direct 
impact  of  lost  availability.  Further,  extended  MTTR  clearly  implies  added  expendi* 
tures  of  costly  manpower  and  test  resources. 

TABLE  2-1  TESTABILITY  CHARACTERISTICS 


TESTABILITY  ASPECT _ 

o  Thoroughness  and  ease  of 
Condition  Monitoring 

-  Fault  Detection 

-  Fault  Isolation 


-  Functional  Verification 


_ SIGNIFICANCE _ 

o  Testing  is  essential  to  full  system  effective¬ 
ness 

o  Operators  need  to  know  the  status  of  system 
operating  modes  with  full  assurance 
o  Valid,  accurate,  unambiguous  detection  and  iso¬ 
lation  of  faults  are  key  to  achieving  maximum 
operational  availability 

o  Functional  test  is  necessary  to  verify  adequacy 
of  performance  before  and  after  maintenance 


o  Constraints  of 

-  Elapsed  Time 

-  Simplicity  of  access 

-  Human  resources 

-  Test  materials 

-  All  cost-generating 
elements 


Testability  discipline  in  all  aspects  has  heavy 
influence  on  the  costs  of  operating  and  support¬ 
ing  prime  mission  equipment  systems 
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TABLE  2-2  TESTABILITY  APPROACHES 


TESTABILITY  REQUIREMENTS 

SIGNIFICANCE 

Inject  into  earliest  designs 

o  Inherent  testability  is  embedded  in  hardware 
design 

o  Late  incorporation  generates  extra  costs 

Provide  active  representation 
in  all  program  life  cycle 
phases 

o  Testability  goals  are  established  and  monitored 
o  Testability  program  plan  to  achieve  goals  at 
minimum  cost  is  developed,  and  maintained 

Testability  posture  at  end  of 
of  each  phase  must  be  set 
to  enter  the  next  phase 

o  Testability  as  a  discipline  is  similar  to 
reliability  and  availability  in  that  it  should  be 
also  considered  in  DSARC  and  like  reviews 

Apply  testability  to  all 
hardware  indenture  levels 

o  Production  requires  bottom-up  integration 
testing 

o  Operational  and  maintenance  requires  top-down 
testing. 

Apply  at  all  maintenance 
levels 

o  Maximise  availability 
o  Minimise  resource  consumption 

2.1.2  The  Role  of  the  Testability  Hof  book 

The  Testability  Notebook  concept  is  to  coalesce  and  codify  extant  literature  and 
knowledge  into  tasks  and  procedures  organised  by  procurement  program  phase.  The 
notebook  provides  a  roadmap  for  defining  and  satisfying  testability  needs  in  each 
phase.  Although  the  Testability  discipline  is  not  yet  fully  formalised,  there  is  no 
general  lack  of  testability  information  and  data.  Much  information  exists,  varying 
from  theoretical  dissertations  to  detailed  descriptions  of  methods  applicable  to 
certain  elements.  Analysis  and  consolidation  of  auch  data,  together  with  opinions, 
techniques,  and  methods  gathered  in  interviews  has  resulted  in  the  Testability 
Notebook  which  codifies  the  information  for  ready  use. 

2. 1.2.1  Testability  Notebook  Concept.  Table  2-3  (page  2-4)  illustrates,  in  summary 
form,  the  basic  concept  taken  for  the  codification  of  tasks  by  development  phase. 
Note  that  the  testability  tasks  relative  to  each  phase  are  addressed  in  terms  of 
both  technical  and  management  practi^s  required.  Systematic,  standardised,  and 
consistent  practices  have  been  identified  or  derived  as  appropriate  to  each  phase  of 
the  life  cycle.  The  practices  Include  procedures,  tools,  tradeoffs,  and  other 
methods  making  up  the  techniques,  and  the  means  employed  to  manage  and  control 
achievement  of  objectives  as  can  be  seen  no  new  or  unique  system  engineering  or 
management  concepts  are  required  for  an  effective  testability  program.  This  same 
general  pattern  applies  to  testability  as  applies  to  other  system  engineering  com¬ 
ponents. 


TABLE  2-3  TESTABILITY  NOTEBOOK  CONCEPT 


Testability  Tasks 
Conceptual  Studies 


Task  Performance 
Techniques 


Manageaent/Control 

Techniques 


e  Establish  functional 
requirements 
e  Establish  goals 
e  Outline  program  plan 


Validation  Phase 


e  Analysis  of  system  utili¬ 
sation  &  support  concept 
e  Analysis  of  performance 
requirements 
e  Close  interface  with 
related  disciplines 


e  Approve  goals 
e  Approve  plan/program 
concept 

e  Establish  criteria 
for  next  phase  entry 


e  Complete  program  plan 
e  Inject  testability 
into  evolving  design 
e  Select  test  system 
e  Plan  ED/FSD  activi¬ 
ties  in  detail 


e  Extensive  coordination 
with  related  disciplines 
and  program  management 
e  Input  to  system  speci¬ 
fications 

e  Develop  test  subsystem 
specifications 


e  Approve  plan 
e  Include  testability 
in  design  reviews 
e  Approve  specifica¬ 
tions 

ie  Establish  criteria 
I  for  next  phase  entry 


Engineering/Full-Scale 
Development  Phase 


e  Develop  test  element 
detail  specifications 
e  Develop  test  system 
elements 

e  Train  human  resources 
e  Demonstrate  goal 

achievement _ 


[e  Growth  of  prior  stage 
activities 

e  Parallels  prime  mission 
equipment  techniques 
e  Operate  proofing/testing 
plans 


e  Approve  specifications 
e  Witness  and  approve 
demonstrations 
e  Establish  criteria 
for  next  phase  entry 


Production  Phase 


e  Produce  test  system 
elements 

e  Deliver  and  deploy 
elements 

e  Assure  smooth  transi¬ 
tion  to  service  use 
e  Institute  information 
feedback 

Deployment/Operatlons 

Phase 


e  Parallels  prime  mission 
equipment  production/ 
delivery/initiation 
techniques 

e  Monitor  initial  use 

e  Assist  users 


e  Include  testability 
indicators  in  moni¬ 
toring  initial  opera 
tional  effectiveness 


e  Monitor  performance 
e  Improve  test  system 
e  Control  changes 


e  Collect/analyse  opera¬ 
tional  data 

e  Sustain  representation 
in  configuration  control 


e  Use  operation  and 
utilisation  standards 
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TABLE  2-2  TESTABILITY  APPROACHES 


TESTABILITY  REQUIRBttNTS 

SIGNIFICANCE 

Inject  into  earliest  designs 

o  Inherent  testability  is  embedded  in  hardware 
design 

o  Late  incorporation  generates  extra  costs 

Provide  active  representation 
in  all  program  life  cycle 
phases 

o  Testability  goals  are  established  and  monitored 
o  Testability  program  plan  to  achieve  goals  at 
minimum  cost  is  developed,  and  maintained 

Testability  posture  at  end  of 
of.  each  phase  must  be  set 
to  enter  the  next  phase 

o  Testability  as  a  discipline  is  similar  to 
reliability  and  availability  in  that  it  should  be 
also  considered  in  DSARC  and  like  reviews 

Apply  testability  to  all 
hardware  indenture  levels 

o  Production  requires  bottom- up  integration 
testing 

o  Operational  and  maintenance  requires  top-down 
testing. 

Apply  at  all  maintenance 
levels 

o  Maximize  availability 
o  Minimize  resource  consumption 

2.1.2  The  Role  of  the  Testability  notebook 

The  Testability  Notebook  concept  is  to  coalesce  and  codify  extant  literature  and 
knowledge  into  tasks  and  procedures  organized  by  procurement  program  phase.  The 
notebook  provides  a  roadmap  for  defining  and  satisfying  testability  needs  in  each 
phase.  Although  the  Testability  discipline  is  not  yet  fully  formalized,  there  is  no 
general  lack  of  testability  information  and  data.  Much  information  exists,  varying 
from  theoretical  dissertations  to  detailed  descriptions  of  methods  applicable  to 
certain  elements.  Analysis  and  consolidation  of  such  data,  together  with  opinions, 
techniques,  and  methods  gathered  in  interviews  has  resulted  in  the  Testability 
Notebook  which  codifies  the  information  for  ready  use. 

2. 1.2.1  Testability  Notebook  Concept.  Table  2-3  (page  2-4)  illustrates,  in  summary 
form,  the  basic  concept  taken  for  the  codification  of  tasks  by  development  phase. 
Note  that  the  testability  tasks  relative  to  each  phase  are  addressed  in  terms  of 
both  technical  and  management  practices  required.  Systematic,  standardised,  and 
consistent  practices  have  been  identified  or  derived  as  appropriate  to  each  phase  of 
the  life  cycle.  The  practices  include  procedures,  tools,  tradeoffs,  and  other 
methods  making  up  the  techniques,  and  the  means  employed  to  manage  and  control 
achievement  of  objectives  as  can  be  seen  no  new  or  unique  system  engineering  or 
management  concepts  are  required  for  an  effective  testability  program.  This  same 
general  pattern  applies  to  testability  as  applies  to  other  system  engineering  com¬ 
ponents  . 
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TABLE  2-3  TESTABILITY  NOTEBOOK  C01 


Testability  Tasks 


Conceptual  Studies 


e  Establish  functional 
requirements 
e  Establish  goals 
e  Outline  program  plan 


Validation  Phase 


e  Complete  program  plan 
e  Inject  testability 
into  evolving  design 
e  Select  test  system 
e  Plan  ED/PSD  activi¬ 
ties  in  detail 


Task  Performance 
Techni 


•ED 


Engineer ing/Eull-Scale 


Development  Phase 


e  Develop  test  element 
detail  specifications 
e  Develop  test:  system 
elements 

e  Train  human  resources 
e  Demonstrate  goal 
achievement 


Production  Phase 


e  Produce  test  system 
elements 

e  Deliver  and  deploy 
elements 

e  Assure  smooth  transi¬ 
tion  to  service  use 
e  Institute  information 
feedback 


Phase 

e  Monitor  performance 
e  Improve  test  system 
e  Control  changes 


e  Analysis  of  system  utili¬ 
sation  6  support  concept 
e  Analysis  of  performance 
requirements 
e  Close  interface  with 
related  disciplines 


e  Extensive  coordination 
with  related  disciplines 
and  program  management 
e  Input  to  system  speci¬ 
fications  , 
e  Develop  test  subsystem 
ifications 
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e  Growth  of  prior  stage 
activities 

e  Parallels  prime  mission 
equipment  techniques 
e  Operate  proofing/testing 
plans 


e  Parallels  prism  mission 
equipment  production/ 
delivery/initiation 
techniques 

e  Monitor  initial  use 

e  Assist  users 


Management/Control 

Te 


EiilESdLP 


e  Approve  goals 
e  Approve  plan/program 
concept 

e  Establish  criteria 
for  next  phase  entry 


e  Approve  plan 
e  Include  testability 
in  design  reviews 
e  Approve  specifica¬ 
tions 

e  Establish  criteria 
for  next  phase  en 


e  Approve  specifications 
e  Witness  and  approve 
demonstrations 
euEstablish  criteria 
for  next  phase  entry 


e  Include  testability 
indicators  in  moni¬ 
toring  initial  opera¬ 
tional  effectiveness 


e  Collect/analyze  opera¬ 
tional  data 

e  Sustain  representation 
in  configuration  control 


e  Use  operation  and 
utilisation  standards 
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2. 1.2. 2  Existing  Guides  to  Testability  Techniques.  The  testability  tasks  of  each 
phase  can  be  defined  by  consideration  of  well-known  and  documented  test  guides  as  is 
done  for  maintainability  and  reliability  programs  via  handbooks,  manuals,  and  pam¬ 
phlets  generated  by  various  service  agencies.  Such  as: 

(1)  RADC  TR-79-327  "An  Objective  Printed  Circuit  Board  Testability  Design  Rat¬ 
ing  System" 

(2)  RADC-TR-79-309  "BXT/External  Test  Figures  of  Merit  and  Demonstration  Tech¬ 
niques" 

(3)  TR-3826  "A  Framework  for  Designing  Testability  into  Electronic  Systems", 
Naval  Surface  Weapons  Center 

(4)  ASD-TR-79-5013  "BIT/SIT  Improvement  Project" 

Extracts  from  and  reference  to  applicable  documents  are  contained  in  the  Notebook 
content. 

2. 1.2. 3  Testability  Application  and  Development  in  Program  Phases.  During  each 
program  phase  different  but  related  tasks  of  appropriate  scope  have  to  be  addressed. 
As  a  particular  example,  in  the  conceptual  studies  phase,  a  preferred  method  of 
establishing  testability  goals  might  be  documented.  A  related  interface  with  reli¬ 
ability  assessment  could  also  be  identified.  Sub-allocations  of  times-to-repair 
(in  establishing  the  maintainability  goal  or  specified  value  of  MTTR)  might  be  made 
to  levels  of  test.  The  availability  discipline  may  be  interfaced  to  aid  in  estab¬ 
lishing  test  concepts  for  scheduled  and  unscheduled  maintenance.  Carrying  this 
example  to  the  full  scale  development  phase,  standardized  procedures  could  be  docu¬ 
mented  to  measure  the  degree  of  achievement  of  test  times  and  allocations.  In  the 
deployment/operations  phase,  standardization  can  be  extended  to  collection  of  field 
data  independently  or  in  conjunction  with  AFM  66-1-based  programs,  or  similar  pro¬ 
grams  of  Army  and  Navy  Services. 

2.2  Definition  of  Life  Cycle  Testability  Functions  and  Tasks  Making  up  the  Notebook 

As  indicated  previously,  established  system  development  phasing  was  taken  as 
the  baseline  or  driving  set  of  events  to  which  all  supporting  disciplines  are  to  be 
keyed.  The  five  program  phases  for  grouping  events  were  taken  as  Conceptual, 
validation.  Full  Scale  Development,  Production  and  Deployment/Operation. 

2.2.1  A  Simplified  Version  of  the  Baseline  Phase  Arrangement  of  Testability  Func¬ 
tions,  Tasks,  and  Interfaces 

In  order  to  provide  an  overview  of  the  content  of  the  phased  arrangement  of 
testability  functions,  tasks  and  interfaces,  the  following  simplified  baseline  for 
the  organization  of  a  testability  program  over  the  five  program  phases  is  provided. 
The  compendia  which  follow  this  section  provide  detailed  engineering  and  management 
guidance  to  the  testability  acquisition  process  for  and  during  each  phase. 

2. 2. 1.1  Testability  Program  Activities  in  the  Conceptual  Phase.  The  basic  concep¬ 
tual  phase  program  activities  are  to  conduct  system  feasibility  studies,  including 
identification  of  alternatives;  to  establish  technical,  military,  and  economic 
bases  for  acquisition  and  to  decide  whether  or  not  to  pursue  the  program.  It  is 
necessary  to  consider  testability  concepts  in  this  phase  because  of  the  weight  their 
consideration  contributes  to  the  decision  process,  and  to  overall  program  costs. 
Table  2-4  summarizes  fundamental  testability  factors  that  most  appropriately  should 
be  accounted  for  during  the  conceptual  phase.  The  testability  relation  to  other 
disciplines  is  indicated  in  the  summary  data  in  Table  2-5. 
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TABLE  2-4  TESTABILITY  OPPORTUNITIES  IN  THE  CONCEPT  PHASE 


Establish  Testing  Concept 

Outline  a  Testability  Program 

Define  functional  testing  requirements 

-  BIT/BITE  versus  external  test  equipment 

-  Test  concepts  at  hardware  indenture  levels  (match 
existing  hardware  concept) 

-  Test  concepts  at  maintenance  levels 
—  Organizational 

—  Intermediate 
—  Depot 

Establish  qualitative/quantitative  testing  goals 

-  Thoroughness  of  condition  monitoring 

-  Time  to  detect  (isolate) 

-  Time  to  complete  functional  test 

-  Man  hours  allocation 

-  Cost  allocation 

-  Management  exception  trigger  level 

-  Testability  figure  of  merit/achievement  goals/thresholds 


TABLE  2-5  CONCEPTUAL  PHASE  TESTABILITY  INTERFACES  WITH  OTHER  DISCIPLINES 


Discipline 
e  Reliability 


e  Maintainability 


e  Design 


e  Availability 

e  Life  Cycle  Cost/ 
Design  to  Cost 
e  Supportability 


•  Management 


Testability  Relation 

-  Failure  rates  -  test  rates  allocations 

-  Critical  areas  -  thoroughness  of  test _ 

-  Allocation  of  access  to  test  points 

-  MTTR  allocations 

-  Functional  design  for  maintainability 

-  Functional  design  for  testability 

-  Accommodation  of  reliability/maintainability/ 
testability  requirements 

-  Design/BIT  ratio  allocation 

-  Test  time  allocation 

-  Scheduled  vs  unscheduled  testing  concept 

-  Costs  of  alternative  testing  approaches 

-  Compatibility  of  testing  approach  alternatives 
with  logistic  support  practices  of  procuring/ 
using  agency 

-  Recognition  of  testability  as  a  program  element 

-  Need  for  management  exception  trigger  levels  and 
allocation 


2 .2. 1.2  Testability  Activities  in  the  Validation  Phase.  The  key  testability 
activity  of  the  Validation  Phase  is  the  creation  of  a  detailed  testability 
program  plan.  The  plan  sets  clear  criteria  for  the  achievement  of  testability 
goals  and  objectives  in  the  subsequent  program  phases.  This  activity  supports 
the  main  program  purpose  of  the  validation  phase  which  is  to  prepare  the 
system  development  concept.  This  phase  is  the  optimum  point  for  the  test¬ 
ability  program  outline  to  be  filled  in  and  refined  by  progressive,  inter¬ 
active  techniques.  The  related  system  and  testability  activities  during 
validation  are  outlined  in  Table  2-6. 

These  testability  activities  contribute  to  the  system  definition,  particularly 
as  requirements  in  the  system  specifications.  An  outgrowth  of  the  testability 
program  definition  is  the  development  of  specifications  for  test  systems,  and 
a  preliminary  listing  of  test  equipment  and  test  resource  requirements.  As 
system  design  detail  fills  in,  the  BIT/BITE  versus  external  test  allocation 
is  refined  over  deeper  levels  of  indenture.  Similarly,  qualitative  and  quan¬ 
titative  testability  measures  and  aims  are  more  closely  related  to  specific 
functional  areas  and  element,  and/or  to  specific  indentures  as  each  particular 
design  situation  warrants.  This  brings  up  the  need  for  a  study  to  determine 
the  best  and  most  economic  methods  and  tools  for  testing  assemblies  of  complex 
devices  such  as  microprocessors,  BOMS,  RAMS,  PROMS  and  other  VLSI/WLSI 
devices  of  current  and  foreseeable  technologies. 


_ TABLE  2-6  VALIDATION  PHASE  ACTIVITIES _ 

•  Basic  Program  Activities 

-  Prepare  System  Development  Concept 

-  Define  program  objectives 

-  Define  program  issues 

-  Detect  potential  special  problems  in  logistic  or  other  support 
disciplines 

-  Define  performance  and  cost  parameters 

-  Assess  program,  cost  schedule  performance  risks 

-  Identify  system  alternatives 

-  Develop  strategy  for  system  and  support  element  acquisition 

-  Perform  breadboard  testing  and  verification 

•  Testability  Activities 

-  Input  testability  requirements  into  system  specifications 

-  Develop  a  preliminary  testability  program  plan 

-  Develop  preliminary  testing  system  specifications 

-  Refine  BIT/BITE/external  test  allocations 

-  Develop  preliminary  listing  of  test  equipment  and  other  test 
resource  candidates 

-  Allocate  quantitative/qualitative  goal  factors  to  functional  areas 
and  elements 

-  Define  preliminary  test  procedural  requirements 
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2 . 2 . 1 . 3  Relationship  With  Other  Disciplines  During  the  Validation  Phase.  The 
maintenance  of  direct  interactions  with  other  disciplines  such  as  reliability, 
maintainability,  and  system  design  is  essential  to  the  development  of  a  superior 
testability  program.  During  the  validation  phase  the  opportunities  for  inter¬ 
action  between  testability  and  other  design  disciplines  increase  rapidly  in 
expanse  and  in  depth  of  detail.  Table  2-7  outlines  these  relationships  and 
some  of  the  more  significant  impact  areas. 


Of  great  importance  at  this  stage  is  the  interface  with  system  and  subsystem 
design  engineers  in  the  applications  of  complex,  diff icult-to-test  technologies. 
Detailed  planning  during  this  phase  ensures  availability  of  the  testing  cap¬ 
abilities  and  facilities  that  will  be  required  in  the  following  development, 
production,  deployment,  and  operations  phases.  Opportunities  must  be  exploited 
at  this  stage  to  optimally  allocate  weight,  space,  and  power  to  BIT,  condition 
monitoring  in  general,  and  maintenance/ test  functions.  In  addition  to  the 
relationship  between  reliability  and  testability  allocations,  testability 
aspects  must  directly  consider  failure  modes  and  effects  and  critical  items. 


Major  interfacings  occur  between  maintainability  and  testability  because  of 
their  very  close  interdependence.  In  many  respects  both  consider  the  same 
elements  but  in  different  aspects.  Some  measures  of  testability  are  also 
measures  of  maintenance  actions,  particularly  time-to-fault-detect  and  time- 
to-fault-isoiate.  Availability  is  also  directly  related  to  both  of  these 
other  disciplines  because  of  the  need  for  and  consumption  of,  system  time  to 
perform  some  actions. 

The  logistics  system  and  testability  have  a  number  of  points  of  interface,  the 
more  important  being  availability  of  support  facilities  and  equipment,  life 
cycle  and  desiyn-to-cost  aspects,  supportability  characteristics,  and  the  main¬ 
tenance  plan. 

Definition  and  resolution  of  the  interdisciplinary  problems  between  testability 
and  interfacing  disciplines  is  a  major  objective  of  the  testability  notebook. 
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TABLE  2-7  VALIDATION  PHASE  DISCIPLINARY  INTERFACES 


Interfacing  Discipline 

Testability  Concerns 

e  Reliability 

-  Failure  rates  allocation  in  greater  depth 

-  Failure  modes  and  effects 

-  Critical  items 

e  Maintainability 

-  Maintenance  measures 
—  MTTR 

—  MDT 

—  MMH 

—  Access  Time 

-  Accessibility 

-  Physical  and  functional  hardware  design 

e  Design* 

-  Condition  monitoring  techniques 

-  Allocation  of  circuitry  to  BIT 

e  Availability 

-  Time  to  test 

-  Scheduled  versus  unscheduled  test 

-  Condition  monitoring  techniques 

e  Life  Cycle  Cost/ 

Design  to  Cost 

-  Test-related  costs 

-  Test-driven  costs  and  alternatives 

e  Supportability 

-  Compatibility  with  logistic  support  planning 
and  practices 

e  System  Test 

-  Coordinate  planning  and  feedback 

-  Software  requirements  and  cost  allocations 

-  Hardware  interface  requirements  and  cost 
allocations 

e  Program  Management 

-  Recognition  of  need  for  testability 

-  Refinement  of  management  exception  trigger 
levels 

♦The  design  interface  also  accommodates  certain  of  the  concerns  listed  with 
other  interfacing  disciplines. 

2. 2. 1.4  Activities  and  Interfaces  in  the  Engineering  Development  and  Full 
Scale  Development  Phases.  The  main  purpose  of  the  engineering  development/ 
full  scale  development  phase  is  to  design  and  develop  the  system,  giving  due 
consideration  for  maintainability,  logistic,  and  testability  factors.  The 
related  system  and  testability  activities  are  outlined  in  Table  2-8.  Test¬ 
ability  activities  in  this  phase  consist  primarily  of  proofing,  polishing  and 
refining  the  testability  concept  and  testability  program  plan  developed  in 
the  preceding  phases.  Feedback  of  information  from  applying  the  plan  in  test 
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programs  and  demonstrations  allows  it  to  be  fine-tuned  to  fulfill  the  require¬ 
ments  of  the  production  and  deployment  phases. 

Specific  activity  occurs  in  the  development  of  detail  specifications  for  test 
equipments  and  resources  and  their  acquisition  proof  testing.  Test  procedures 
are  developed  and  proofed  using  prototype  and  development  model  hardware, 
which  in  turn  enables  determination  of  the  achievement  of  testability  measure¬ 
ment  goals.  Particular  refinements  are  being  made  during  this  phase  in 
activities  related  to  logistics  support,  calibration,  and  maintenance  of  test 
equipment,  and  to  facilities.  These  activities  are  now  extended  from  strict 
system  application  to  consideration  of  system  support  facilities.  Another 
refinement  is  the  activity  to  demonstrate  by  measurements  the  achievement  of 
assigned  testability  goals.  The  development  of  test  procedures  for  specific 
items  of  hardware  is  the  culmination  of  tasks  associated  with  selection  of 
test  equipment,  definition  of  test  requirements,  hardware  design,  and  other 
test  resources.  These  specific  instances  are  reflections  of  the  intensified 
activity  to  completely  define  the  testability  concept  and  the  plan  in  all  of 
its  aspects. 


The  culmination  of  all  this  effort  during  this  phase  is  a  fully-refined  and 
proofed  testability  program  plan,  firm  testability  concept,  and  a  completely- 
ready  test  system.  These  three  testability  elements  must  be  ready  for  active 
use  in  the  final  production,  deployment,  and  operation  phases. 


TABLE  2-8  ENGINEERING  DEVELOPMENT/FULL  SCALE  DEVELOPMENT  ACTIVITIES _ 

•  Program  Activities 

-  Develop  Unit  Fabrication  Specifications 

-  Develop  all  Specifications  in  full  detail 

-  Design  complete  system 

-  Design  and  test  hardware  (DTE/OTE) 

-  Develop  logistic  support  planning  and  elements 

-  Demonstrate  Achievement  of 
—  Performance  parameters 

—  System  and  support  operational  goals 
—  System  and  support  utilization  goals 

-  Initiate  system  configuration  control 

-  Develop  modifications  planning  system 

•  Testability  Activities 

-  Develop  specifications  for  test  elements  and  resources 

-  Develop  and  proof  test  elements  and  resources 

-  Develop  and  validate  test  procedures 

-  Develop  logistics  and  calibration  support  for  test  elements 

-  Develop  and  conduct  cadre  test  and  maintenance  training 

-  Develop  configuration  planning  for  test  system,  keep  consistent  test 
system  (with  PME) 

-  Develop  test  software  configuration  control  procedures 

-  Demonstrate  achievement  of  testability  goals 

-  Finalize  planning  for  test  in  production,  operator  &  deployment  phases 

-  Modify  testability  plan  to  reflect  all  changes _ 


In  the  engineering  development/full  scale  development  phase  the  interactions, 
impacts  and  influences  between  testability  and  other  design  characteristics 
reach  peak  activity  levels  .  Close  coordination  and  cooperation  by  personal 
direct  consultations  is  needed  to  keep  all  design  elements  moving  ahead  in 
phase  with  one  another  and  with  schedule  milestones.  All  persons  involved  in 
the  total  system  concept  must  be  aware  at  all  times  of  the  essential  purpose 
of  their  combined  effort:  the  achievement  of  the  system  goals  of  operational 
readiness,  effectiveness,  and  supportability  on  a  production  basis  in  the 
deployment/operational  arena.  Interface  activities  shown  in  Table  2-9  are 
mainly  a  further  intensification  of  those  in  the  validation  phase,  with  the 
growth  that  is  to  be  expected  as  the  details  of  system  definition  are  filled 
in.  It  is  important  in  this  phase  that  the  testability  engineer  has  a  position 
on  the  configuration  control  board  to  ensure  that  testability  features  are  not 
neglected  when  considering  modifications,  changes  and  redesigns.  It  is  also 
during  this  phase  that  feedback  from  the  test  and  evaluation  portion  of  the 
development  activities  begins  to  emerge.  Positive  action  will  enable  the  most 
effective  use  of  this  data,  particularly  where  results  indicate  that  corrective 
action  is  needed.  Hence,  this  information  is  an  important  key  to  smoothing 
and  polishing  the  interdisciplinary  problems.  Closed  loop  problem  solution 
procedures  are  required  in  all  disciplines  including  testability. 


TABLE  2-9  TESTABILITY  INTERFACES  IN  ENGINBBRING/FULL  SCALE  DEVELOPMENT 

•  Largely  the  same  as  in  the  validation  phase  but  at  greatest  intensity, 
depth,  detail,  and  personal  contact  levels,  e.g.: 

-  System  Specification 

-  BIT/BITE  Test  Allocation 

-  Reliability 

-  Maintainability 

-  Availability 

•  Provide  feedback  from  test  and  evaluation  activities 
e  Establish  a  voice  in  configuration  control 


2. 2. 1.5  Activities  and  Interfaces  in  the  Production/Deployment  Operational 
Phases.  In  the  production/deployment/operation  phases,  the  prime  mission 
equipment  and  its  supporting  elements  are  produced,  positioned  in  using 
location,  and  placed  into  operational  use.  Since  these  phases  normally  over¬ 
lap,  sustaining  support  for  fielded  equipments  gradually  passes  from  the 
manufacturer  to  the  acquiring/using  agencies,  until  past  the  end  of  production 
when  the  user/maintainer  has  full  responsibility  for  the  sustaining  effort. 

Table  2-10  opposite  lists  the  main  testability  activies  and  the  discipline 
interface  function  in  these  phases.  The  major  activities  involve  application 
and  evaluation  of  the  plan  and  its  constituent  elements.  A  substantial  portion 
of  this  activity  is  directed  toward  improving  performance  by  enforcement  of 

2-11 


standards  and,  where  needed,  procedural  changes  or  improvements  in  supporting 
hardware,  software,  or  facilities.  Associated  with  this  is  the  sustaining 
effort  to  ensure  that  testability  requirements  are  given  due  consideration  in 
all  design  changes,  whether  instituted  for  producibility  purposes  or  to  improve 
system  performance.  The  testability  engineer  must  retain  his  position  on  the 
change  control  board.  Another  significant  testability  task  of  these  phases  is 
that  of  evaluating  lessons  learned  and  their  incorporation  into  new  and  ongoing 
testability  programs. 

TABLE  2-10  ACTIVITIES  AND  INTERFACES  OF  THE  PRODUCTION  AND  DEPLOYMENT/ 

OPERATIONS  PHASES* 


Program  Activities 

•  Prime  Mission  Equipment  System  and  supporting  elements  are  produced  and 
positioned  for  operational  use. 

•  At  close  of  production  phase  sustaining  support  gradually  passes  to 
Government  acquiring,  operating,  and  support  agencies. 

e  Systems  and  supporting  elements  are  utilized  in  operational  activities. 
Testability  Activities 

•  Monitor  testability  element  performance. 

e  Obtain  feedback  to  compare  performance  to  program  standards  and  goals. 

•  Improve  performance  through  enforcements  and/or  procedure  changes  and/or 
prime  mission  equipment/Support  Hardware  modifications. 

•  Improve  performance  through  requisite  maintenance/modification  of  test 
software. 

•  Incorporate  lessons  learned  into  new  or  in  process  testability  programs. 
Testability  Interfaces 

e  Largely  the  same  as  in  the  Engineering  Development/Full  Scale  Development 
Phase  but  at  much  reduced  intensities  and  with  much  lessened  personal 
contact,  e.g. : 

-  System  Test  Effectiveness 

-  Personnel  Skill  Demands 

-  Reliability/Maintainability  Feedback 

•  Government  using  and  supporting  agencies 


•NOTE:  Activities  of  deployment/operations  phase  may  be  Government 
responsibility,  depending  upon  contract  provisions  and  the 
transition  plan. 
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2.3  Guide  for  Using  the  Notebook 

The  results  of  the  Testability  Notebook  Development  are  documented  in 
testability  program  flow  charts,  testability  task  compendia,  and  application 
notes.  The  compendia  (with  flow  charts),  and  the  material  described  previously 
(including  a  master  flow  chart  and  a  description  of  the  compendium  format)  is 
contained  in  the  sections  which  follow. 

2.3.1  Testability  Program  Flow  Charts 

The  flow  charts  serve  as  an  index  to  the  testability  tasks  and  represent 
the  overall  philosophy  of  program  flow  derived  from  the  analysis.  A  composite 
flow  chart,  covering  all  phases,  is  presented  in  Figure  2-1.  Each  phase  is 
broken  into  functional  activity  areas,  shown  as  numbered  blocks;  each  block 
encloses  the  tasks  of  that  activity.  Relationships,  interfaces,  interactions 
and  flows  among  functional  activity  areas  and  tasks  are  represented.  The 
complex  interactions  between  non-adjacent  functions  may  be  directly  connected 
by  arrows;  or  may  be  shown  by  small  arrowed  rectangles  containing  individual 
numerals  which  correspond  to  the  connecting  function.  The  flow  charts  provide 
the  logical  flow  for  the  testability  activities  and  tasks  (and  their  necessary 
inputs)  to  be  undertaken  during  each  phase,  and  the  relationships  among  such 
activities  and  tasks  from  one  phase  to  the  next.  As  such  they  provide  end  to 
end  guidance  as  to  when  each  activity  or  task  is  relevant  and  identifies  their 
prerequisite  inputs. 

The  flow  for  the  conceptual  phase  illustrates  the  mechanization.  Blocks 
Cl,  C2  and  C3  are  functional  areas  containing  1,  3  and  1  tasks  respectively. 
Each  task  is  identified  by  a  title  and  a  reference  number.  Unnumbered  inputs 
to  the  blocks  signify  data  accessed  from  outside  the  testability  discipline. 
Results  of  function  C2  are  used  in  functions  V 2  and  V3.  Functions  Cl,  C2  and  C3 
all  connect  to  function  VI  (the  first  sequential  activity  of  the  Validation 
phase) . 

In  function  V2  task  V2A  is  artificial.  Its  purpose  is  to  present  a  check 
list  of  design  guidance  appropriate  to  the  general  task  of  injecting  testabil¬ 
ity  into  the  design.  (Features  of  the  check  list  which  are  specifically 
appropriate  to  other  tasks,  e.g.,  controllability  and  observability,  V2F  and 
V2G,  are  repeated  in  those  task  compendia.) 

The  placement  of  the  Critical  Design  Review  in  the  Full  Scale  Development 
flow  exemplifies  the  generalization  of  the  flow  charts.  In  any  particular 
system  program,  there  will  be  preferential  ways  of  phasing  events  which  will 
supersede  previous  general  guidance.  The  CDR  might  thus  occur  earlier  or  later 
in  the  phase,  and  the  performance  of  any  related  task  would  need  adjustment  of 
its  inputs  and  outputs  to  achieve  the  objectives.  The  arrangement  of  the  tasks 
and  task  compendia  is  intended  to  support  the  needed  flexibility. 

In  the  Compendia  Sections  of  the  Notebook,  the  program  flow  is  repeated  in 
the  form  of  an  individual  flow  chart  for  each  phase. 
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Conceptual  Phase  Testability  Tasks 


RELATEO  FUNCTIONS 


Validation  Phase  Testability  Tasks 


TESTABILITY  PROGRAM 
PLAN  PREPARATION 


VI 


A.  DESCRIBE  THE  T  PROGRAM 
TASKS 

B.  PREPARE  TESTABILITY  PRO¬ 
GRAM  MILESTONES 

C.  IDENTIFY  THE  f  RESPONSI¬ 
BILITIES,  AUTHORITY,  AND 
INTERFACES 


E} 


/ 


PERFORMANCE 

WEIGHT 


SIZE _ 

RELIABILITY 


LCC 


PERFORMANCE 
SUPPORT  ABILITY 


A.  TRADE-OFF  ALTERNATE 
OESIGNS  OF  BIT.  EXTERNAL 
AUTOMATIC  TEST,  AND 
MANUAL  TEST  MIXES 


INHERENT  TESTABILITY 
ANALYSIS  OF 

PRELIMINARY  DESIGN 

SYSTEM  DESIGN 

TESTABILITY 

INCORPORATION 


V2 


A.  GENERAL  T 
GUIDANCE 


DESIGN 


B.  PROVIDE  FOR  DESIGN  COM¬ 
PATIBILITY  OF  UNITS  TO  BE 
TESTED  WITH  SELECTED  OR 
AVAILABLE  ETE  OR  ATE 

C.  PROVIDE  DIRECT  DESIGNS 
AND  TEST  SEQUENCES  USING 
PARTS  SELECTED  FOR 
TESTABILITY 

D.  DESIGN  UNITS  WITH  TEST¬ 
ABLE  PHYSICAL  PARTITIONING 

E.  INCORPORATE  INITIALIZATION 
INTO  DESIGN 

F.  INCORPORATE  CONTROL¬ 
LABILITY  INTO  DESIGN 

G  INCORPORATE  OBSERV¬ 
ABILITY  INTO  DESIGN 

H.  DETERMINE  THE  NUMBER  AND 
PLACEMENT  OF  UUT  TEST 
POINTS 

I.  INCORPORATE  SYSTEM  LEVEL 
BIT 


A.  ASSESS  THE  EXTENT  OF 
QUALITATIVE  TESTABILITY 
INCLUDED  IN  DESIGN 

B.  DEFINE  FJGURES-OF-MERIT 
FOR  INHERENT  T 

C.  ANALYZE  HW/SW  BIT 
FEATURES 

D.  CONDUCT T“ANALYSIS  OF 
THE  POTENTIAL  UUTS  IN 
THE  PRELIMINARY  DESIGN 


TESTABILITY 

ANALYSIS 

REPORT 
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PREPARE  TESTABILITY 
ANALYSIS  REPORT 


Production  Phase  Testability  Tasks 


Deployment  Phase  Testability  Tasks  — •»  < 


PROM  FULL  SCALE 
DEVELOPMENT 
PHASE 


MONt  TOR/EVALUATE 
TESTABILITY  DATA 


A.  MONITOR  THE  PRODUCTION 
PROCESS  AND  TRENDS, 
REVIEW  CHANGE 
PROPOSALS 


TO 

DEPLOYMENT 

PHASE 


MONITOR/EVACUATE 
TESTABILITY  DATA 


A.  MONITOR  THE  OPERATION 
AND  SUPPORT  ACTIVITY 

B.  REVIEW  CHANGE  PRO 
POSALS  FOR 
TESTABILITY 


Figure  2-1.  Testability  Program  Task  Flow. 


2.3.2  Testability  Task  Compendium  Format  and  Characteristics 

Each  Testability  Task  Compendium  consists  of  the  following  segments i 
TASK  REFERENCE  NUMBER: 

Identifies  the  task  by  number  and  traces  directly  to  the  Testability 
Flow  Diagram. 

PHASE: 

Identifies  the  task's  position  in  the  procurement  cycle.  It  traces 
directly  to  the  Testability  Flow  Diagram. 


FUNCTION: 

Identifies  the  name  of  the  activity  block  (the  top  line  of  each  block 
in  the  flow  diagram)  which  includes  this  task. 

TASK  TITLE: 

Gives  the  name  of  the  particular  task.  The  title  is  also  placed 
in  the  flow  diagram  with  the  task  reference  number. 


TASK  OBJECTIVE: 

Provides  a  concise  statement  describing  the  goal(s)  of  the  task. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

Lists  the  design  groups  the  testability  engineer  will  coordinate 
within  the  task's  performance,  the  input  data  he  will  need  from 
the  design  groups  and  the  output  expected  from  him. 

COST  TRADEOFF  INTER-RELATIONSHIPS: 

Describes  the  equipment  liife  cycle  costs  to  be  considered  in 
performing  and  managing  the  task. 

TASK  SYNOPSIS: 

Consists  of  three  parts 

(1)  Task  Requirements,  which  describe  what  has  to  be  done  to 
acconplish  the  task. 

(2)  Implementation,  which  describes  the  design  techniques, 
tradeoff  tools,  optimization  techniques  and  technical 
guidance  needed  to  accomplish  the  task. 

(3)  Conpletion  Criteria,  which  provides  a  check  list,  figure- 
of-merit  or  other  criteria  for  use  in  judging  the  completion 
of  the  task. 


For  some  tasks,  certain  detailed  tools  and  techniques  have  been  placed  in 
appendices  to  the  individual  compendium. 


Source  material,  where  used,  has  in  general  been  subjected  to  editing  to  fit 
the  style  and  format  of  the  task  compendia.  In  most  cases,  the  technical 
context  and  content  of  the  source  material  has  been  preserved. 


2-15 


2.3.3  Application  Notes 

Three  considerations  should  be  noted  in  using  the  contents  of  the  Testability 
Notebook . 

(1)  Most  importantly,  the  system/unit  designers  themselves  are  to  be  en¬ 
couraged  to  be  testability  engineering  conscious.  It  is  far  more  preferential 
that  the  design  engineer  be  well  acquainted  with  testability  (and  of  course 
reliability,  maintainability  and  supportability)  design  techniques  as  he  is 
'with  performance  design  techniques,  than  for  him  to  "answer”  to  or  be  driven 
by,  a  somewhat  alien  testability  engineer.  However,  most  designers  need  assis¬ 
tance,  and  they  should  be  given  the  testability  assist  -  ahead  of  their  actual 
design  effort  -  in  terms  of  guidance  for  inclusion  rather  than  requirements  for 
correction  of  an  already  developed  design.  It  is  not  proper  to  take  the 
approach  of  achieving  testability  via  ECP. 

(2)  There  is  a  broad  spectrum  of  possible  program  activity  and  task  phas¬ 
ing,  and  that  presented  in  the  Testability  Notebook  should  be  taken  as  guidance 
only.  Rearrangement  of  task  phasings,  inputs  and  outputs  may  well  need  to  be 
accomplished  for  many  programs  which  either  have  bypassed  certain  phases  or  for 
which  testability  engineering  has  been  omitted  from  earlier  phases. 

(3)  The  full  scope  of  all  source  data  has  been  necessarily  reduced  to  fit 
the  Notebook.  To  have  included  full  scope  would  have  been  self-defeating  In 
terms  of  bulk  and  added  confusion.  The  condensations  in  scope  have  been 
accomplished  in  three  ways  (additional  to  wording  redundancy  among  sources); 
these  are  by  editing  into  appendix  format,  by  condensing  for  inclusion  In  the 
task  synopsis  and  by  noting  for  reference  only.  Notebook  users  are  therefore 
encouraged  to  obtain  copies  of  source  data  where  added  scope  of  treatment 
appears  helpful. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  ClA 


PHASE: 


Conceptual 


FUNCTION : 


TASK  TITLE: 


TASK  OBJECTIVE: 


Testability  Requirements 

Establish  qualitative  and  quantitative  testability 
requirements  for  prime  system. 

Establish  testability  requirements  that  are  based  upon 
operational  requirements  in  sufficient  time  to  guide  the 
preliminary  system  design  effort. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS : 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


•  System 
Engineer 

•  Maintainability 
Engineer 


•  System  Functional  Concept 


•  Preliminary  Maintenance 
Concept,  Maintenance 
Facilities 


Preliminary  T 
Requirements 

Preliminary  T 
Requirements 


Logistics 

Support 


•  Logistics  System,  Manning 


•  Safety  Engineer  •  Safety  Considerations 


•  Preliminary  T 
Requirements 

•  Preliminary  T 
Requirements 


•  Program  Manager  •  Critique/approval  of  T 

Requirements 


•  Preliminary  T 
Requirements 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Timely  development  of  the  testability 
requirements  defined  herein  will  simplify  subsequent  tasks  and  is  necessary 
to  assure  minimum  program  life  cycle  costs. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 


1.1  Translate  the  operational  readiness  and/or  equipment  availability 
requirement  into  the  following  testability  requirements:^ 
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a. 


Maximum  allowable  time  between  the  occurrence  of  a  failure  condition 
and  the  reporting  of  the  failure  (failure  latency)  for  each  mission 
function. 


b.  Degree  of  failure  tolerance  required  for  each  mission  function. 

c.  Maximum  system  downtime  due  to  corrective  maintenance  actions  at  the 
organizational  level. 

d.  Testing  requirements  of  backup  (standby)  equipment  and  functions  in 
order  to  accommodate  system  degraded  mode  requirements. 

2.  Task  Implementation. 

Establishing  the  testability  requirements  is  an  iterative  process  in 
which  the  testability  requirements  cure  optimized  with  respect  to  other 
system  characteristics,  e.g.,  BIT/ATE  utilization,  manual/autcmatic 
test  equipment  for  system  monitoring,  and  optimizing  the  mix  of  BITE, 
portable  testers  and  maintenance  shops  to  support  organizational 
maintenance .  The  testability  requirements  established  by  this 
iterative  process  form  the  basis  for  the  system  specification  testa¬ 
bility  requirements.  Subsequently,  functions  C2  and  C3  include  tasks 
which  treat  (respectively)  tradeoffs  among  the  requirements/ character¬ 
istics  and  the  merging  the  testability  requirements  into  the  specifica¬ 
tion. 

2.1  The  qualitative  and  quantitative  testability  requirements  should: 

a.  factor  safety  considerations  into  the  requirements  for  failure 
;  detection  and  failure  tolerance. 

b.  be  based  upon  expected  numbers  and  skills  of  operating  and 
maintenance  personnel. 

c.  be  consistent  with  constraints  imposed  by  the  logistic  system, 
including  GFE  support  systems, 

d.  be  consistent  with  the  preliminary  maintenance  concept,  deployment 
scenarios,  environmental  conditions  and  planned  maintenance 
facilities . 

2.2  The  evaluation  of  operational  considerations  is  the  starting  point  for 

identifying  system  requirements.^) 

o  Review  operational  concept-threat,  mission  analysis,  operating 
environment. 

o  Determine  prime  function  of  the  system  and  relate  it  to  the 
operational  concept. 


1-4 


r 


p  Determine  operational  limiting  constraints.  These  parameters 
establish  the  maximum  repair  time  that  enable  all  operational 
scenarios  to  be  accomplished. 

-  Availability 

-  Critical  failure  mode 

-  Operational  fault  detection/fault  isolation  CFD/FI ) 

-  Failure  latency  time 

-  Isolation  level 

-  Allowable  downtime 

2.3  Many  factors  affect  testability  and  should  be  kept  in  mind  during  the 
tradeoff  process  throughout  the  program.  ^  The  important  factors 
relating  to  operation  and  maintenance  scenarios  are: 

o  Mission  Success 

-  Reliability 

-  Confidence  Level  (Failure  Detection  Probability,  Minimum  False 
Alert) 

o  Avai labi li ty 

-  Readiness  Test  Time 

-  Maintainability 

oo  MTTR 

-  Required  Skill  Level  (Training  Time  and  Cost) 

-  Complexity  (Modularization,  Standardization) 

-  Fault  Isolation  Time  (ATE  Software  and  Machine  Loading  Time, 

BIT  Costs;  Manpower) 

-  Fault  Correction  Time 

-  Verification  Time  (Operator  Time,  BIT  and  ATE  Software/ 

Hardware  Costs) 

-  Test  Recycling  Time  Due  to  Non-Detectable  Faults  (Manpower, 

ATE  Machine  Loading  Time) 


o  Logistics 

-  Weight,  Size,  Shape 

-  Mission  Duration 

-  Distance  from  Supply  Source 

-  Storage  Limitations 

-  Software,  Manuals,  Tools  Required 

-  Manpower  Requirements 

-  Test  Instrumentation  Maintenance,  Tools 

-  Test  Recycling  Time  Due  to  Non-Detectable  Faults  (Manpower,  Test 

Instrumentation) 
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o  Expansion  Capability  of  System  (BIT ,  ATE) 
o  Life  Cycle  Costs 

-  Hardware 

-  Software 

-  Maintenance 

-  Administrative 

3.  Completion  Criteria. 

3.1  The  task  is  completed  when  there  is  agreement  between  testability 

engineering,  the  interfacing  groups  and  the  program  manager  that  the 
iterative  process  has  established  the  optimum  prime  system  testability 
requirements . 


\ 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  C2A 


PHASE: 


FUNCTION : 

TASK  TITLE: 

TASK  OBJECTIVE: 


Conceptual 

Tradeoffs 

Establish  testing  levels  for  BIT/BITE  and  ETE  testing 

Identify  the  boundary  or  level  to  which  BIT/BITE  testing 
will  be  utilized. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


•  System 
Engineering 

•  Design 
Engineering 


•  System  Level  BIT/BITE/ETE 
Approaches 

•  Hardware  (HW)  BIT/BITE 
Alternatives 


•  T  Guidelines 


•  T  Guidelines 


•  Application 
Software 


•  Software  (SW) 
Alternatives 


T  Guidelines 


•  Life  Cycle 
Costing 

•  Maintainability 
Engineering 


•  Tradeoff  Costing  Assistance  • 


•  Proposed  Maintainability 
Program  Plan  and  Integrated 
Logistics  Support  Plan 


Costing  Data  and 
Parameters 

T  Guidelines 


COST  TRADEOFF  INTER-RELATIONSHIPS:  The  recurring  and  nonrecurring  costs 
associated  with  various  test  mixes  need  evaluation  in  this  task.  Generally, 
software  approaches  (BIT  and  ATE)  exhibit  competitive  acquisition  costs  but 
lower  maintenance  costs  (including  ECP  impacts)  compared  to  approaches  which 
rely  more  heavily  on  unique  test  systems. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

The  task  is  performed  to  identify  the  boundary  or  level  to  which  BIT/BITE 
testing  will  be  utilized. 
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1.1  BIT/B1TE/ETE  utilization.  The  norm  is  to  establish  a  two-level, 

preferably  automatic,  test  concept  which  is  driven  by  the  natural 
break  between  BIT  capabilities  and  ETE  capabilities. 

a.  Built-in  test  should  be  utilized  to  provide  initial  fault  detection 
for  a  system  or  equipment  and  to  provide  initial  fault  isolation 

to  a  small  group  of  modules. 

b.  Off-line  or  External  Test  Equipment  (preferably  approved  ATE)  should 
be  used  to  provide  fault  detection  for  a  module  as  a  Unit  Under  Test 
(UUT)  and  to  provide  fault  isolation  to  components  within  a  module. 

1.2  BIT/BITE/ETE  Coordination. ^  Strive  for  maximum  utilization  of  the 
system  BIT  capability  which  is  distributed  to  individual  UUTs  in 
determining  off-line  test  requirements  for  the  UUTs. 

1.3  Timeliness  Requirement.  System  BIT/BITE/ETE  definition  must  be 
accomplished  early  in  the  conceptual  phase.  The  following  paraphrased 
excerpt  illustrates  the  cost  effectiveness  of  early  and  proper 
decisions:  (4) 


"The  total  cost  of  a  sample  weapons  system  over  its  life  cycle- 
approximate  ly  20  years  or  100,000  hours  -  has  an  interesting  cost 
parameter  distribution.  The  engineering  validation  pha  a  requires 
only  3  to  4  percent  of  total  life-cycle  cost,  while  the  preproduc¬ 
tion  phase  takes  12  percent  -  a  total  of  just  15  percent  for  the 
entire  prototyping/preproduction  period.  Production  costs  typically 
average  35  percent;  operation/support  costs  comprise  50  percent 
(Fig  1) .  The  17-22  year  ownership  of  the  system,  not  the  initial 
acquisition  expense,  represents  the  primary  cost  factor. 


"Further  DoD  studies  demonstrate  that  once  design  is  complete,  the 
best  ETE  approach  can  reduce  life  cycle  cost  by  at  most  5  percent; 
production  testing  can  achieve  a  10  percent  cost  enhancement, 
and  hardware  design  at  most  15  percent.  Proper  system  design  and 
architecture  with  sufficient  provision  for  built-in  testability 
may  influence  upwards  of  70  percent  of  the  life  cycle  cost." 


1.4  Maintenance  Time  and  Availability  Impacts . 


Early  identification  of  prime  system  characteristics  and  test  subsystem 
characteristics  is  essential  for  the  test  subsystem  to  be  effective  in 
performance  monitoring,  fault  detection,  and  fault  isolation.  Figure  2 
illustrates  these  key  points . 
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Figure  1.  Major  Life  Cycle  Cost  Drivers 


o  Mean  corrective  maintenance  time  (Mct)  is  a  major  determinant  in 
selecting  candidate  test  subsystems. 

o  The  prime  system's  characteristics  are  a  key  determinant  in 
selecting  the  optimum  performance  monitoring  (PM)  and  BIT 
effectiveness  goal. 


2 .  Implementation . 


2.1  BIT/BITE  Selection  Factors 


a.  Prime  equipment  can  be  tested  either  with  BIT  or  with  external  test 
equipment.  For  predominantly  digital  systems,  the  hardware  BIT 
concept  is  the  more  cost  effective  approach.  Factors  reducing  the 
cost  of  BIT  logic  include: 
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State-of-the-Art  (Circa  '75-’8Q)  Prime  System  vs  Test  Subsystem  Characteristics 


(1) 


Efficient  coding  techniques  minimize  the  added  logic  complexity 
for  BIT. 

C2)  Use  may  be  made  of  unfilled  PC  board  package  positions  for 

at  least  a  portion  of  the  BIT  logic,  so  that  the  cost  penalty 
for  BIT  is  considerably  less  than  the  increased  logic 
complexity . 

(3)  Added  design  effort  is  minimal  since  test  constraints 

necessarily  will  be  imposed  and  the  built-in  test  design  is 
accomplished  with  systematic  logic  design  procedures. 

b.  The  BIT  has  the  objectives  of  1)  detecting  subsystem  faults,  2) 
isolating  to  one  subsystem  module  and  3)  providing  aid  in  isolation 
to  a  specific  faulty  I C  component. 

c.  Using  BIT  hardware  instead  of  software,  a  test  method  can  be 
developed  to  fault  detect  and  fault  isolate  to  a  digital  subsystem 
and  to  the  faulty  module  therein.  The  test  procedure  compares 
module  output  patterns  (obtained  by  stimulating  each  module  with 
fixed  input  patterns)  to  the  known  responses  of  fault-free  modules. 
This  test  technique  is  commonly  used  in  programmed  test  equipment 
for  static  testing  of  digital  modules.  Generally,  higher  level 
assemblies  (entire  subsystems)  are  too  complex  for  this  pattern 
comparison  technique  and,  therefore,  require  complex  test  sequences 
and/or  parametric  testing.  If  each  subsystem  is  self-testing  and 
self -isolating  to  the  module  level,  external  test  equipment  and 
even  certain  levels  of  maintenance  can  be  greatly  simplified  or 
eliminated.  External  test  equipment  requirements  may  be  simplified 
to  provide  only  power,  cooling,  initialization,  and,  optionally 
limited  display  or  comparison  of  module  test  results.  Further,  the 
self  test  may  be  performed  in  an  operational  system,  isolating 
operational  faults  not  detected  in  maintenance.  Only  milliseconds 
are  required  for  BIT  performance. 

(7) 

2 . 2  ETE  Selection  Factor s . 


There  are  two  main  functions  which  must  be  performed  in  ETE  concept 
formulation.  These  are  establishing  performance  monitoring  need 
and  determining  the  degree  of  off-line  ETE  at  organizational, 
intermediate,  and  depot  levels. 


a.  There  are  three  subsets  of  the  Performance  Monitoring  Needs  to 
be  considered: 

o  The  level  to  which  system  and  platform  performance  monitoring 
will  be  performed. 

o  The  environmental  factors  (e.g.,  safety,  damage  control,  EMCON- 
Electromagnetic  Emission  Control)  to  be  monitored. 
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o  The  system  configurations  which  need  to  be  displayed  e.g., 
communication  channels,  electrical  power  plant  status). 

From  the  information  available  at  this  stage  of  development,  it 
is  possible  to  start  formulating  the  performance  monitoring  needs 
of  the  system.  Such  things  as  BIT,  BITE,  ON-Line  Test,  and  Self 
Test  needs  can  be  formulated.  These  needs  can  then  be  blended  into 
the  definition  of  the  prime  system  and  the  support  system.  This 
effort  will  produce  a  preliminary  integrated  prime  hardware  and 
support  system  design,  Subsequent  iterations  with  the  other  factors/ 
operators  in  the  definition  process  may  change  the  hardware  definition 
but  certain  general  attributes  remain  constant.  It  is  quite  feasible 
to  define  BIT/BITE  or  On-Line  Test  during  the  first  cut  at  a  hardware 
definition  (design)  and  have  it  remain  relatively  stable  through 
the  development  cycle.  Because  of  the  highly  integrated  nature  of 
BIT/BITE  and  the  prime  hardware  it  is  almost  mandatory  that  it  be 
specified/defined  during  the  conceptual  phase  of  development.  It  is 
too  late  to  wait  until  the  full  scale  development  or  production 
phases  to  introduce  BIT/BITE  or  On-Line  ETE.  Performance  monitoring 
needs  should  also  be  identified  early  in  the  development  cycle  so 
that  the  requisite  sensors  and  monitoring  points  can  be  designed/built 
into  the  prime  hardware.  The  decision  to  use  full  or  partial  off-line 
ETE  can  and  should  also  be  made  this  early  in  the  development  cycle, 
to  allow  an  evolutionary  selection  process. 


b.  In  addressing  the  degree  of  off-line  ETE  for  each  maintenance  level, 
a  variety  of  iterations  and  the  tradeoffs  are  possible  with  impact 
on  the  full  range  of  ETE  concepts.  This  tradeoff  of  alternative 
concepts  is  a  factor  in  the  definition  of  ETE  alternatives.  Following 
are  the  steps  taken  in  the  definition  of  alternatives. 

(1)  Propose  Generic  ETE  Types.  Generic  ETE  types  are  proposed 
which  are  compatible  with  each  maintenance  concept  under 
study  as  a  first  step  in  the  selection  process.  Possible 
generic  ETE  options  for  a  system  can  be  made  up  from  one  or 
a  combination  of  the  following: 

o  Built-In  Test  (BIT) 

o  Built-In  Test  Equipment  (BITE) 

o  Other  on-line  test  systems 

o  Off-line  test  systems 

(2)  Identify  On-Line  Test  Requirements.  Prime  system  and  mission 
needs  will  have  to  be  analyzed  to  determine  the  need  for 
on-line  monitoring.  In  general,  from  an  operational  and 
maintenance  viewpoint,  on-line  monitoring  (or  testing)  is 
the  most  desirable  mode  of  operation.  The  tradeoffs 
involved  are  cost,  technical  impact  on  the  design,  and 

the  operational  requirements  for  the  mission. 
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(3)  Identify  The  Degree  of  Off-line  Test  Requirements.  Many  of  the 
selection  considerations  for  on-line  test  requirements  are  also 
applicable  to  off-line  testing.  The  degree  of  off-line  testing 
will  largely  depend  on  decisions  regarding  specific  maintenance 
levels  and  locations  (i.e.,  organizational,  intermediate  and 
depot) . 

At  the  conceptual  level  of  development  the  ETE  alternatives  can 
only  be  matched  with  the  degree  of  detail  available  on  the 
prime  hardware,  its  cost  estimate,  and  the  technical  defini¬ 
tion  of  detail  of  the  hardware  alternatives.  It  is  quite 
likely  that  several  possible  acceptable  alternative  ETE 
concepts  will  still  exist  after  the  end  of  the  concept  phase. 

2.3  Documentation . 

The  aim  of  this  task  is  to  provide  the  most  efficient  use  of  and  the 
most  cost  effective  mix  of  BIT/BITE  and  ETE  over  the  program  life 
cycle.  A  plan  for  the  level  of  BIT  within  the  system  design  and  the 
ETE  required  should  be  developed.  «  The  plan  may  contain  more  than  one 
alternative  that  meets  the  operational  requirements .  Selection  of  the 
optimum  mix  (alternative)  will  come  about  as  the  design  firms  up  in 
later  phases. 

3.  Completion  Criteria. 

The  task  is  completed  at  government  approval  of  the  documented  plan  for 
achievement  of  the  optimum  BIT/BITE/ETE  mix. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  C2B 


PHASE: 


Conceptual 


FUNCTION : 


Tradeoffs 


TASK  TITLE:  Perform  manual  test  equipment  vs  ATE  tradeoffs  for  each 

maintenance  level. 

TASK  OBJECTIVE :  Define  the  mix  of  Manual  and  Automatic  Test  Equipment  at 

each  maintenance  level. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS : 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


•  Design 

Engineering 


•  System/Equipment  Require¬ 
ments,  BIT  Capability  Data 


•  HW  Testability 
Requirements 


•  Test  Engineering  •  Test  Complexity  Factors 


•  Testability  Requirements 


•  Test 

Equipment 

Engineering 


•  Manual  Test  Equipment  and 
ATE  Capabilities,  Require¬ 
ments,  &  Costs  Data 


•  Test  Equipment  Testabil¬ 
ity  Requirements 


•  Application 
Software 


•  SW  Program  Requirements 


•  SW  Testability  Require¬ 
ments 


•  Maintainability 
Engineering 


•  Maintainability  Program  •  Testability  Requirements 

Concept,  Integrated  Logistics 
Support  Concept 


•Life  Cycle 
Costing 


•  Costing  Guidelines  and  Cost  •  Cost  parameters 
Analysis  Results 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Non-recurring  and  recurring  costs  for 

acquisition,  use  and  logistic  support  of 
test  equipment  and  tesv.  programs  are  affected  by  this  decision.  Benefits  and 
penalties  occurring  in  all  phases  from  Validation  through  Operations  and 
Support  must  be  considered. 


TASK  SYNOPSIS: 


1.  Task  Requirements. 

Develop  an  integrated  test  policy  for  the  system,  trading  use  of  manual 
versus  use  of  automatic  test  equipment  (ATE)  for  each  maintenance  level. 
Consider  test  complexity,  repair  policy,  fault  isolation  time,  function¬ 
al  test  time,  operational  environment,  logistic  support  requirements, 
development  time,  skill  levels,  and  all  acquisition  and  ownership  costs. 

2.  Implementation . 

2.1  Test  Tradeoffs Decisions  regarding  the  type  of  test  equipment  to  be 
used  for  system  monitoring  and  maintenance  should  be  made  based  upon 
repair  policies  and  overall  maintenance  plans.  Tradeoffs  should  be  made 
for  test  requirements  at  each  maintenance  level,  considering  test 
complexity,  time  to  fault  isolate,  operational  environment,  logistic 
support  requirements,  development  time  and  cost.  The  degree  of  testing 
automation  should  be  consistent  with  the  planned  skill  levels  of  the 
equipment  operators  and  maintenance  personnel. 

2.2  Considerations .  Include  the  following  considerations  in  the  tradeoffs, 
where  applicable: 

a.  Costs  to  buy  or  develop  test  equipment  (T.E.) 

b.  Skill  level  of  personnel  required  to  support  T.E. 

e.  Development  time  for  selected  T.E. 

d.  Adaptability  of  T.E.  to  design  changes 

e.  Manning  requirements  of  T.E. 

f.  Utilization  of  T.E. 

g.  Programming  requirements  and  costs  of  T.E. 

h.  UUT  fault  isolation  and  repair  time  using  T.E. 

i.  T.E.  failure  rates,  fault  isolation  requirements  and  time,  and 

repair  time 

j.  Total  LCC  of  selected  T.E, 

k.  Ability  of  T.E.  to  meet  system  test  requirements 

l.  Prime  system  availability  and  maintainability  requirements 

m.  Special  tester  and/or  interface  requirements  for  system  UUT's  such 

as  micro-processor  and  hybrid  boards 

n.  Test  equipment  failure  rates  (availability) 

o.  Other  contractually  specified  requirements 

2.3  Test  Equipment  Costs.  Tradeoffs  should  evaluate  the  proposed  test 
methodologies  for  total  LCC.  This  evaluation  should  include  initial 
price  (hardware,  software,  adapters,  and  patchboards),  programming  time, 
future  test  requirements,  and  system  throughput.  The  form  shown  in 
Table  l^8)  provides  a  form  of  basic  guidance  which  is  adaptable  for  use 
in  cost  comparisons  of  various  systems. 
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Either  an  existing  or  a  candidate  new  system  may  be  set  up  as  a  base¬ 
line  for  comparison  purposes.  Acquisition ,  use  and  support  costs  should 
be  segregated  for  separate  viewing,  so  that  choices  may  be  between 
competing  systems  on  the  basis  of  equivalent  life  cycle  cost  elements, 
individual  and  total. 


TABLE  1.  CALCULATION  OF  COST  OF  OWNERSHIP 


Section  I  -  System  acquisition 


A  Jy*t  A/ 
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(a)  Hardware  costs 


(b)  Software  costs 


(c)  Support  costs 


Total  System  Acquisition  Costs 


atchboards,  fixtures , etc. 


(b)  Test  program  cost 


(c)  Set-up  cost 


(d)  Test  cost 


(e)  Fault  isolation  cost  (troubleshoot) 


(f)  Set-up  revest  cost 


(g)  Retest  cost 


Total  production  test  cost 


Section  III  -  Data  analysis  and  reports 


(a)  Quality  control  reports 


(b)  Logistics  and  field  service  reports 


(c)  Configuration  management  reports 


Total  data  analysis  and  report  costs 


Section  IV  -  other  ownership  costs 


(a)  Refurbishment  cost 


(b)  Multistation  capabilit 


(c)  Remote  station  control 


Total  other  ownership  costs 


TOTAL  COST  OF  OWNERSHIP 


Achievement  Definitions.  Successful  completion  of  this  task  will  result 
in  selection  of  the  most  cost  effective  (in  terms  of  total  LCC)  test 
system.  In  the  conceptual  phase ,  the  selection  process  may  not  achieve 
selection  of  a  singular  system  but  should  at  least  define  a  peculiar 
type  of  test  system,  leaving  the  final  selection  for  validation  phase. 
The  type  of  system  chosen  should  be  documented,  along  with  the 
rationale  used  to  arrive  at  the  chosen  system  for  evaluation  in  the 
validation  phase. 

Completion  Criteria. 


The  task  is  completed  upon  Government  concurrence  in  the  documented 
selection  of  test  system. 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  C2C 


PHASE; 


FUNCTION ; 


Conceptual 

Tradeoffs 


TASK  TITLE: 


Develop  unit  design  features  compatible  with  selected 
or  available  ETE. 


TASK  OBJECTIVE: 


Control  prime  equipment/ETE  interface  compatibility  and 
associated  costs. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER  RELATIONSHIPS: 


T  INTERFACE  INPUT  TO  T 


OUTPUT  FROM  T 


•  System 
Engineering 

•  Design 
Engineering 

•  Test  Equipment 
Engineering 

•Life  Cycle 
Costing 


•  Performance  Data 


•  UUT  Interface 
Data 

•  ETE  Interface 
Data 

•  LCC  Data  and 
Requirements 


•  ID  Related  Requirements,* 
and  Mechanizations 

•  Compatibility  Mechaniza¬ 
tions 

•  Data  Supporting  LCC 


*  Ideally,  the  interface  should  be  established  between  the  design  and  test 
equipment  engineering  functions,  with  monitoring  only  by  system  and 
testability  engineering. _ 


COST  TRADEOFF  INTER-RELATIONSHIPS:  The  control  of  the  unit/test  equipment 

interface  requires  some  added  non-recur¬ 
ring  effort  in  design  and  engineering.  Reduced  efforts  as  a  result  occur 
downstream  in  both  non-recurring  and  recurring  costs  of  interface  devices, 
adapters,  procedures,  technical  data,  training  and  manpower  expended  in 
maintenance  test  and  repair.  The  cost  breakeven  occurs,  and  savings 
begin  to  accrue,  during  validation  or  full  scale/engineering  development 
phases . 


TASK  SYNOPSIS: 

1.  Task  Requirements. 


1.1  The  requirement  for  compatibility  control  applies  to  all  external  test 
equipment ,  whether  manual  or  automatic,  general  or  special.  The  concept 


of  control  is  applied  to  minimize  the  complexity  and  cost  of  maintenance 
testing,  without  degrading  the  operational  performance  of  the  prime 
system.  In  this  synopsis,  the  term  ETE  can  be  construed  to  include 
automatic,  manual,  general  or  special  purpose  test  equipment  external 
to  the  BIT/BITE  within  the  prime  system.  Controls  are  established  in 
the  conceptual  phase  to  effect: 

a.  Control  of  the  design  of  prime  equipment  interfaces  to  meet 
existing  ETE  interface  specifications. 

b.  Control  of  the  design  of  new  ETE  to  meet  prime  equipment  interface 
specifications . 

2.  Implementation . 

Published  compatibility  techniques  deal  primarily  with  automatic  test 
equipment,  but  are  generally  readily  applicable  to  other  types  of  test 
equipment . 

2.1  Fundamental  Considerations.  Policy  imposed  by  the  acquisition  agency  or 
program  manager  may  set  the  mode  of  the  task  by  designating  either  that 
existing  test  equipment  be  utilized  or  that  new  test  equipment  should  be 
designed.  In  any  event,  some  combination  of  three  overall  methodologies 
will  probably  be  applicable  to  any  particular  project : ^ 

a.  Control  of  the  design  of  prime  equipment  interfaces  to  meet 
existing  ETE  interface  specifications. 

b.  Control  of  the  design  of  new  ETE  to  meet  prime  equipment  interface 
specifications . 

c.  Design  of  interface  devices  to  bridge  incompatible  prime  equipment 

and  ETE.  ' 

(As  a  first  priority,  it  is  usually  most  effective  to  utilize  ETE 
systems  existing  in  inventory  or  already  under  development  which  will 
meet  suppjort  requirements . ) 

2.2  The  following  is  a  checklist  that  is  applicable  to  compatibility  tasks. 
The  checklist  is  stated  in  the  form  of  questions  relating  to  oommonly 
recognized  ETE  compatibility  design  parameters.  ^ 

a.  ETE  COMPATIBILITY  CHECKLIST 

(1)  System  Modularity.  Is  the  system  functionally  modularized  to 
the  UUT  level  of  asaembly/disassembly? 

(2)  Functional  Independence .  Are  the  system  and  its  UUT’s  capable 
of  being  tested  without  stimulation  by  another  system  or  UUT 
and  without  simulation  of  another  system  or  UUT? 
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(3)  Adjustments.  Are  system/UUT  adjustments  (e.g.,  trimming, 
tuning-,  alignment)  required  while  testing  on  ETE?  (An 
adjustment  includes  any  action  that  changes  variable 
components  such  as  potentiometers,  variable  capacitors, 
inductors,  transformers,  etc.,  that  affect  operation  of  the 
equipment . ) 

(4)  Ancillary  External  Test  Equipment.  Is  additional  external 
equipment  required  to  generate  a  stimulus  or  to  monitor 
response  signals? 

(5)  Environmental.  Will  the  system  or  the  UUT's  require  special 
environmental  considerations  during  test  on  the  ETE,  such 

as  vacuum  chambers,  oil  baths,  shake  tables,  ovens,  cooling 
air  and  screen  rooms? 

(6)  Stimulus  and  Measurement  Accuracies.  Are  the  stimulus  and 
measurement  accuracies  required  for  high  confidence  test 
available  in  the  candidate  ETE? 

(7)  Test  Point  Adequacy.  Are  sufficient  test  points  provided  for 
non-ambiguous  fault  isolation  and  for  monitoring  redundant 
circuits  and  BIT  circuits? 

(8)  Test  Point  Characteristics.  Are  test  point  impedance  and 
voltage  levels  compatible  with  the  ETE  interface? 

(9)  Test  Point  Isolation.  Will  damage  to  a  UUT  result  from  a 
short  circuit  between  any  test  point  and  ground?  will  wideband 
noise  impressed  on  the  test  point  degrade  performance? 

(10)  Power  and  Load  Requirements.  Are  the  current  and  voltage 
required  for  system/UUT  power  and  the  loads  required  to 
absorb  the  output  power  available  at  the  ETE? 

(11)  Warm-Up.  Will  the  system  and/or  any  UUT  require  warm-up  on 
ETE  to  ensure  accurate  test? 

(12)  Access .  Is  internal  access  adequate  for  visual  inspection  and 
manipulative  actions? 

(13)  Packaging.  Is  access  to  UUT  adequate?  Can  the  UUT  be  removed 
and  replaced  easily?  Are  special  tools  required? 

(14)  Safety  (Personnel) .  Will  the  maintenance  action  require 
personnel  to  work  under  hazardous  conditions  such  as  close 
proximity  to  high  voltage,  radiation,  moving  parts,  or 
high-temperature  components,  etc. 

(15)  Connector  Standardization.  Are  standard  connectors  used  on 
the  system/UUT? 


(16)  Connector  Keying  and  Accessibility.  Are  the  system/ UUT 
connectors  keyed  to  preclude  inserting  any  connector  into  the 
wrong  receptacle,  and  are  they  readily  accessible  for  quick 
connection  and  removal? 

(17)  EMI  or  RFI .  Will  the  system  require  special  testing  due  to 
EMI  or  RFI  performance  characteristics? 

2.3  Design  Objective; 

/ 

Successful  completion  of  this  task  results  in  a  proposed  UUT/ETE 
interface  design  that  is  compatible  with  both  the  prime  system  and 
the  ETE.  Interface  devices  (adapters)  should  be  held  to  a  minimum 
in  quantity  and  complexity. 

3.  Completion  Criteria. 

Documentation  should  be  developed  describing  the  test  interface  with 
justification  for  the  designs  and  test  equipments  chosen.  The  task 
is  complete  when  the  government  acquisition  manager  accepts  the  design 
factors  and  candidate  equipment. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  C3A 


Conceptual 

Preliminary  System  Specification 

Incorporate  testability  requirements  in  preliminary 
system  specification 

Refine  the  requirements  developed  in  Task  Number  CIA 
into  specifiable  goals  for  test  and  testability 
requirements  and  include  the  results  in  the  preliminary 
system  specification. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS : 


T  INTERFACE  INPUT  TO  T  OUTPUT  FROM  T 


•  System  •  System  Requirements  •  Testability  Requirements 

Engineering  and  Tradeoffs 


COST  TRADEOFF  INTER-RELATIONSHIPS;  There  are  no  significant  cost  impacts  in 

the  specification  input  task.  Over¬ 
specification  should  be  avoided  because  of  obvious  cost  ramifications,  but 
the  depth  of  testability  specified  must  be  adequate  to  avoid  subsequent  costly 
re-design  because  of  field  test  and  maintenance  conqplexities. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  testability  goals/requirements  should  include,  but  not  be  limited 
to,  the  following  subjects 

a.  Requirement  for  status  monitoring. 

b.  Definition  of  the  failure  modes  specified  to  be  the  basis  for  test 
design. 

c.  Requirement  for  failure  detection  (failure  coverage,  failure  latency) 
using  full  test  resources. 

d.  Requirement  for  failure  detection  using  built-in  test  resources. 


PHASE: 

FUNCTION : 

TASK  TITLE: 

TASK  OBJECTIVE: 
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e.  Requirement  for  failure  detection  using  only  passive  monitoring. 

f.  Requirement  for  limiting  false  alarm  rate. 

g.  Requirement  for  failure  localization  to  a  subsystem/ equipment  using 
built-in  test 

h.  Requirement  for  failure  isolation  to  one  or  more  number  of  modules 
using  built-in  test.  The  requirement  may  .^e  expressed  in  terms  of 
percentage  of  modules  in  a  subsystem/eguipment. 

i.  Requirement  for  failure  localization/isolation  times. 

j.  Restrictions  on  built-in  test  resources  in  terms  of  hardware  size, 
weight  and  power,  memory  size  and  test  time. 

k.  Requirement  for  BIT  hardware  reliability . 

2 .  Implementation . 

Functions  Cl  and  C2,  if  performed,  will  have  provided  baseline  data  for 
use  in  the  specification  input_task.  If  there  has  been  no  prior 
interface,  development  of  the  T  requirements  for  inclusion  in  the 
system  specifications  may  need  to  be  started  on  a  qualitative  level, 
then,  given  the  development  of  interfaces  with  system  design  personnel, 
developed  to  a  quantitative  level. 

(2) 

2.1  System  Specification  Flow  Diagram. 

The  process  for  development  of  the  preliminary  system  specification 
may  be  regarded  as  a  flew  of  tasks  as  depicted  in  Figure  1.  This 
flow  process  is  comprised  of  a  series  of  subtasks  that  upon  completion 
lead  to  completion  of  the  preliminary  system  specification.  Certain 
testability  features  such  as  observability  and  controllability  are 
not  specified  per  se  but  are  built  into  the  system  specifications 
through  the  system  test  requirements,  MTTR  requirements,  and  in  the 
fault  detection  and  fault  isolation  (FD/FI)  requirements.  The  flow 
diagram  in  Figure  1  recapitulates  all  the  testability  functions  and 
tasks  of  the  conceptual  phase. 

2.2  Test  Subsystem  Performance  Specification,  Basic  Testability  Contents 

To  optimize  the  subsystems  performance  and  design,  the  following  prime 
system  and  test  subsystem  performance  parameters  must  receive  due 
consideration  in  developing  the  system  specification: 

a.  Operational  demand  and  criticality 

b.  Mission  duration  and  operational  modes 

c.  Mission  reliability 


k 
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d .  BIT  MTBF 

e.  Maximum  turn-around  time  (as  applicable) 

f.  Allowable  mean  down  time  (as  applicable) 

g.  Expected  mean  logistics  and  administrative  delay  times  (M^  and 

h.  Mean  corrective  maintenance  time  (M  . ) 

ct 

i.  Test  subsystem  effectiveness  (E^) 

j .  Percentage  of  false  alarms ,  no  defect  removals  (FA) 

2.3  Detail  Testability  Requirements. 

The  following  list  of  17  generic  groupings  are  the  BIT/ETE  Figures -of- 
Merit  found  in  a  survey  of  successful  system  specifications. 

a.  fraction  of  faults  detected: 

(1)  percent  of  all  faults  automatically  detected  by  BIT/ETE 

(2)  percent  of  all  faults  detectable  by  BIT/ETE 

(3)  percent  of  all  faults  detectable  on-line  by  BIT/ETE 

(4)  percent  of  all  faults  and  out-of-tolerance  conditions 

detectable  by  BIT/ETE 

(5)  percent  of  all  faults  detectable  by  any  means 

b.  fraction  of  false  alarms 

£ 

(1)  rate  at  which  false  indications  occur  (per  10  hours) 

(2)  percent  of  indicated  failures  caused  by  actual  failures 

(3)  percent  of  BIT/ETE  indicated  failures  caused  by  actual  failures 

(4)  percent  of  BIT/ETE  fault  isolations  to  the  wrong  UUT 

c.  fraction  of  false  status  indications 

(1)  percent  of  erroneous  BIT  indications 

d.  mean  fault  detection  time 

(1)  time  to  indicate  a  fault  once  it  has  occurred 

(2)  time  to  detect  a  fault  once  it  has  occurred 

e .  mean  BIT/ETE  running  time 

(1)  time  to  verify  that  a  failure  has  occurred/or  has  been  repaired 
using  BIT/ETE 

f .  frequency  of  BIT/ETE  executions 

(1)  percent  of  all  equipment  functions  tested 
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g.  test  thoroughness 


(1)  percent  of  all  equipment  functions  tested 

h.  fault  isolation  resolution 


(1)  isolation  of  P^  percent  of  the  failures  to  X^  UUT's,  P2  percent 

of  the  failures  to  X2  UUT's  and  so  on,  with  any  fault  isolation 
method 

(2)  isolation  of  all  faults  to  less  than  or  equal  to  some  maximum 
number  of  UUT's 

(3)  isolation  of  P  percent  of  the  failures  to  X^  UUT's,  P2  percent 
of  the  failures  to  X2  UUT's,  and  so  on,  with  BIT/ETE 

(4)  isolation  of  a  specified  percent  of  the  failures  to  less  than 
or  equal  to  a  specified  quantity  of  UUT's  at  the  various 
maintenance  levels 

(5)  isolation  of  a  specified  percent  of  the  failures  down  to  less 
than  or  equal  to  a  maximum  number  of  plug-in  modules 

(6)  isolation  semi-automatically  to  a  certain  percent  of  all  faults 
down  to  a  specified  number  of  UUT's 

i.  fraction  of  faults  isolated 

(1)  isolate  a  specified  percent  of  failures  that  occur  within  a 
specified  maximum  time 

(2)  isolate  a  failure  down  to  a  replaceable  level,  within  a 
specified  average  time 

(3)  isolate  a  failure  down  to  a  replaceable  level  within  a 
specified  time  once  the  fault  isolation  process  has  been 
initiated 


k.  maintenance  personnel  skill  level 

(1)  all  maintenance  actions  must  be  capable  of  being  performed  by 
a  specified  quantity  of  maintenance  personnel  with  a  specified 
skill  level,  at  various  maintenance  levels 

(2)  BIT/ETE  must  be  designed  for  use  by  a  specified  minimum  skill 
level  technician 


i 
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BIT/ETE  mean-time-to-repair 


(1)  mean-time-to-repair  ETE 

(2)  mean-time-to-repair  monitoring/fault  isolation  functions 

m.  BIT/ETE  mean  time  between  failures 

(1)  mean  time  between  failures  of  monitoring/fault  isolation 
functions 

(2)  mean  time  between  failures  of  ETE  only 

n.  BIT/ETE  availability 

(1)  monitoring/fault  isolation  functions  should  be  operating  with 
a  specified  probability  of  survival 

o.  mean-time-to-repair 

(1)  system/ equipment  MTTR  and  maximum  repair  time 

(2)  system/equipment  MTTR  and  maximum  repair  time  at  various 
maintenance  levels 

P-  availability 

(1)  inherent  availability 

(2)  operational  availability 

q.  active  memory  allocated  for  BIT/ETE  functions 

(1)  monitoring/fault  isolation  functions  shall  take  up  a  specified 
percent  of  active  computer  memory 

2.4  Reference  Examples. 

a.  Reference  (11)  contains  a  sample  on-board  test  system  specification 
for  part  of  the  B1  aircraft's  central  integrated  test  system  (CITS) . 
CITS  provides  on-aircraft  information  relative  to  the  health  of 
sub-systems  by  in-flight  monitoring  and  providing  stimulus  for 
ground  testing. 

b.  Reference  (12)  contains  sample  testability  paragraphs  for  system 
specification. 

3.  Completion  Criteria. 

The  task  is  successfully  completed  by  incorporating  the  testability 
goals/requirements  typified  by  Paragraph  1,  Task  Requirements,  and 
adapting  those  applicable  portions  of  Paragraph  2,  Implementation, 
into  the  preliminary  system  specification.  The  task  is  therefore 
judged  complete  on  government  approval  of  the  preliminary  system 
specification. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  VIA 

PHASE;  VALIDATION 

FUNCTION:  Testability  Program  Plan  Preparation 

TASK  TITLE;  Describe  the  testability  program  tasks 

TASK  OBJECTIVE;  Define  and  describe  the  tailored  tasks,  to  be  pursued 
throughout  the  conduct  of  the  Testability  Program,  as 
part  of  the  Testability  Program  Plan  which  represents  the  overall  testing 
strategy  including  functional  test  and  on-line/of f-line  test  considerations. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS; 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 

e 

Test  requii  ments,  prelimin- 

e  T  Task  Definitions 

Engineering 

ary  system  specifications. 

system  tradeoff  candidates 

e  Design 

e 

Test  requirements,  prelimin- 

e  T  Task  Definitions 

Engineering 

ary  LRU  specifications. 

tradeoff  candidates 

e  Maintainability 

e 

Maintainability 

Engineering 

requirements 

e  Logistics 

e 

Logistic  support  concepts 

Support 

and  constraints 

e  Application 

e 

Diagnostic  software  and 

Software 

BIT  tradeoff  candidates 

e  Life  Cycle 

e 

Baseline  cost  constraints 

Costing 

e  Program 

e 

Program  Master  Schedule, 

e  Testability  Program 

Management 

Testability  Program  Plan 

Plan 

Re vi ew/Cr i t ique/Appr ova 1 

COST  TRADEOFF  INTER-RELATIONSHIPS;  None  directly  applicable.  The  planned 
tasks  will  include  provisions  for  cost  estimating. 


I 


TASK  SYNOPSIS: 


1.  Task  Requirements. 

1.1  The  Testability  Program  Plan  is  a  critical  entry  in  the  CDRL  for  an 
Advanced  Development  contract.  The  plan  describes  the  contractor's 
understanding  of  the  testability  requirement  end  his  approach  to 
implementing  and  enforcing  the  requirement  within  his  organizational 
structure  and  through  his  subcontractors.  The  Testability  Program  Plan 
must  indicate  that  the  contractor  is  giving  adequate  attention  to  support- 
ability  through  logistic  support  analysis  and  testability  analysis  and 
that  he  is  scheduling  adequate  development  time  to  permit  a  testable 
design. W 


1.2  The  following  excerpt  from  a  candidate  Data  Item  Description  (DID)  for 
a  testability  program  plan  contains  the  preparation  instruction  for  a 
testability  program  plan. O-) 

-  The  Testability  Program  Plan  shall  present  the  overall  testing 
strategy  including  operational  checks,  periodic  on-line  tests,  and 
off-line  test  considerations.  It  shall  present  milestones  to  be 
met  to  ensure  that  the  final  design  achieves  the  required  degree 
of  testability.  The  plan  includes  the  mechanisms  for  the  reporting 
of  progress,  problems,  and  tradeoffs,  and  the  enforcement  of  the 
proper  use  of  testability  design  features  by  designers  and  sub¬ 
contractors.  The  plan  shall  include  the  following: 

a.  The  work  to  be  accomplished  for  each  task  delineated  in 
MIL-STD-XXX. (3) 

b.  Program  milestones  and  customer  reviews. 

c.  The  contractor  organizational  element  responsible  for  the 
implementation  of  the  Testability  Program, 

d.  Interfaces  between  that  responsible  organizational  element 
and  related  elements  such  as: 

Systems  engineering 
Design  engineering 
Maintainability  engineering 
Logistics 
Support  equipment 
Training 

Operational  software 
Diagnostic  software 
Maintenance  documentation 
Test 

e.  Control  over  subcontractor  and  vendor  testability  program. 


i 
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2.  Implementation . 


2.1  The  plan  will  define  how,  during  the  validation  phase,  the  following 
function  will  be  performed* 


a.  Incorporate  testability  design 

b.  Perform  BIT  tradeoffs 

c.  Analyze  inherent  testability 

d.  Prepare  testability  analysis  report 

e.  Prepare  development  specifications 

f .  Support  PDR 

2.2  The  plan  will  define  how,  during  the  full  scale  development  phase, 
the  following  tasks  will  be  performed*. 

a.  Perform  test  requirements  analysis 

b.  Analyze  design  test  effectiveness 

c.  Identify  testability  costs/penalties 

d.  Identify  testability  benefits 

e.  Prepare  testability  demonstration  plan 

f.  Conduct  testability  demonstration 

g.  Prepare  final  testability  analysis  report 

h.  Support  Cl  CDRS 

i  Evaluate  operational  testability 
; j .  Support  prime  system  CDR 

2.3  The  plan  will  define  how,  during  the  production  phase,  the  following 
tasks  will  be  performed: 


■'V.. 
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a.  Monitor  production  process  and  trends 

b.  Review  change  proposals 

X'The  plan  will  define  how,  during  the  operation  &  support  phase,  the 
cbm^ractor  will  monitor  organizational/intermediate/depot  testability 
dataT^ 
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2.5  The  Testability  Program  Plan  will  be  contractually  required  by  CDRL/D1D. 

The  degree  of  acceptability  can  be  measured  by: 

a.  Compliance  with  CDRL/DID  requirements 

b.  Comparison  to  the  topic  checklist  in  the  design  requirements 
as  a  gauge  of  completeness 

c.  Contract  data  quality  assurance  function  review  of  the 
completed  plan  for  clarity,  conciseness  and  editing 
principles. 


3.  Completion  Criteria. 

The  task  is  completed  upon  Government  acceptance  of  the  Testability 
Program  Plan. 


i 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V1B 


PHASE: 


VALIDATION 


FUNCTION: 


Testability  Program  Plan  Preparation 


TASK  TITLE: 


Prepare  testability  program  milestones 


TASK  OBJECTIVE:  The  Program  Milestones  show  (1)  the  time  phasing  of  each 

task  and  its  interrelationship  with  other  tasks,  and 
(2)  data  submissions  and  their  review,  verification  and  utilization. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


•  System 
Engineering 


Data  Submissions,  Reviews, 
Hardware  Fabrication  and 
Performance  test  schedules 


•  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


•  Design 
Engineering 


Data  Submissions,  Reviews, 
Activity  Schedules 


•  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


•  Maintainability 
Engineering 


Data  Submissions,  Reviews, 
Activity  Schedules 


•  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


•  Reliability 
Engineering 


•  Data  Submissions,  Reviews, 
Activity  Schedules 


•  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


•  Application 
Software 


•  Data  Submissions,  Reviews, 
Activity  Schedules 


•  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


Life  Cycle  •  Data  Submissions,  Reviews, 

Costing  Activity  Schedules 


e  Testability 
Milestones, 
and  Liaison 


Program 

Schedule 


Program 

Manager 


e  Data  Submissions,  Reviews, 
Activity  Schedules 


e  Testability 
Milestones, 
and  Liaison 


Program 
Schedule , 


COST  TRADEOFF  INTER-RELATIONSHIPS:  None  directly  applicable. 
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TASK  SYNOPSIS; 


1.  Task  Requirements. 

1.1  Each  milestone  is  a  predetermined  point  of  accomplishment  which  is 

clearly  recognizable  as  an  event  which  either  does  or  does  not  occur  at 
a  predetermined  point  in  time;  for  examples  "Complete  fabrication  of 
developmental  model."  Areas  or  phases  known  to  be  potentially  con¬ 
trolling  efforts  or  those  known  to  be  pushing  the  state  of  the  art  should 
be  carefully  identified  and  milestoned. 


2.  Implementation . 

2.1  The  Testability  Milestone  Plan  consists  of  a  series  of  clearly  defined 
milestones  with  the  scheduled  (planned)  time  span  and  completion  date  of 
each.  The  Testability  Milestone  Plan  format  and  symbology  should  be 
standardized  and  consistent  with  other  plans  within  the  program.  Key 
milestone  data  for  the  parent  program  should  also  be  displayed  to  show 
the  interrelationships  with  Testability  activities. 


2.2  The  Testability  Milestone  Plan  should  contain  the  testability  function 

schedules.  The  following  is  a  list  of  testability  activities  that  should 
be  considered  in  formulating  the  milestones. 

a.  Validation  Phase. 

-  Design  Units  to  be  compatible  with  selected  or  available  ATE 

-  Select  parts  for  T;  provide  direct  designs  and  test  sequences 

-  Design  Units  with  testable  physical  partitioning 

-  Incorporate  initialization  into  design 

-  Incorporate  controllability  into  design 

-  Incorporate  observability  into  design 

-  Determine  the  number  and  placement  of  UUT  test  points 

-  Incorporate  system  level  BIT 

-  Incorporate  general  T  design  features 

-  Tradeoff  alternate  designs  of  a  BIT,  external  automatic  test, 
and  manual  test  mix 

-  Characterize  failure  modes  (types) 

-  Estimate  frequency  of  occurrence  of  each  failure  mode 

-  Show  that  qualitative  T  is  included  in  design 

-  Prepare  T  analysis  model  for  each  preliminary  design 

-  Define  Figures-of -Merit  for  inherent  T 
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-  Analyze  HW/SW  BIT  features 

-  Prepare  T  analysis  report 

~  Prepare  Cl  development  specifications 

-  Prepare  Computer  Program  Cl  Development  Specifications  (CPCI) 

-  Prepare  ETE  specifications  or  select  ETE 

-  Support  PDR 

b.  Full  Scale  Development  Phase. 

-  Produce  a  Test  Requirements  document  for  each  UUT 

-  Produce  a  Diagnostic  Software  Specification 

-  Insure  total  testability 

-  Predict  levels  of  fault  detection  and  fault  isolation 
for  the  equipment  and  each  UUT 

-  Identify  development  costs  and  recurring  costs  and 
penalties 

-  Estimate  testability  impact  upon  development/operation 
and  support 

-  Prepare  Testability  Demonstration  Plan 

-  Conduct  Testability  Demonstration 

-  Prepare  Final  Testability  Report 

-  Review  testability  features  and  predicted  testability 
parameters 

-  Monitor /evaluate/propose  corrective  action 

-  Provide  field  testability  data 

c.  Production  Phase. 

-  Monitor  production  process  and  trends.  Review  change 
proposals 

d.  Operation  and  Support  Phase. 

-  Monitor  Organizational,  Intermediate,  and  Depot  level  T  data 

e.  Contractual  dates  such  as  contract  award,  contract  end, 
and  program  reviews 


3.  Completion  Criteria. 


3.1  Task  is  completed  on  government  acceptance  of  the  Testability  Program 
Plan. 
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PHASE: 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Humber  VIC 

VALIDATION 


FUNCTION:  Testability  Program  Plan  Preparation 

TASK  TITLE:  Identify  the  testability  responsibilities,  authority 

and  interfaces 


TASK  OBJECTIVE:  Identify  the  organizational  responsibilities  and  authority 

for  testability  management  including  control  of  subcontracted 
engineering,  levels  of  control  for  design  requirements,  reviews  and  documenta¬ 
tion.  Describe  interfaces  between  the  organizational  element  responsible  for 
testability  and  other  related  elements.  Show  how  adequate  surveillance  will 
be  maintained  to  enforce  all  testability  requirements. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE _ INPUT  TO  T  _ _  OUTPUT  FROM  T 


e  Program 
Manager 

e  Testability  Tasks,  Subcon¬ 
tract  Engineering  Control, 
Design  Requirements  Control, 
Review  Control,  Documenta¬ 
tion  Control 

e  Documented  Acknowledge¬ 
ment  of  Authority  and 
Statements  of 
Responsibilities 

e  Customer 

e  Division  of  Responsibility 
and  Authority 

•  Documented  Acknowledge¬ 
ment  of  Authority  and 
Statements  of 
Responsibilities 

COST  TRADEOFF 

INTER-RELATIONSHIPS:  None 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  following  is  a  list  of  testability  organizational  interfaces. 

a.  System  Engineering 

b.  Design  Engineering 

c.  Maintainability  Engineering 

d.  Logistics  Support 

e.  Support  and  Test  Equipment 


11-11 


f.  Training 

g.  Application  Software 

h.  Maintenance  Documentation 

i .  Production  Engineering 

j .  Reliability  Engineering 

k.  Life  Cycle  Costing 

l.  Safety  Engineering 

m.  Program  Management 

n.  Test  Engineering 

o.  Human  Factors 


2.  Implementation ■ 

2.1  In  all  interfaces,  the  testability  engineer  should  be  prompt,  active  and 
assertive.  Testability  is  most  effectively  instilled  into  a  system  if 
design  guidelines  are  provided  when  design  commences.  It  is  inefficient 
for  the  testability  (or  any  other  -ility)  engineer  to  take  the  role  of  a 
post-design  critic  and  difficult  as  well  as  uneconomical  for  designers 
to  incorporate  changes  to  documented  designs  when  those  changes  reflect 
design  criteria  which  could  have  and  should  have  been  disclosed  during 
or  ahead  of  design  formulation.  Timeliness  is  therefore  a  principal 
responsibility  for  the  testability  engineer. 


2.2  The  testability  engineer's  charter  is  to  establish  and  maintain  an  effec¬ 
tive  testability  program  that  is  an  integral  part  of  the  overall  design 
effort.  In  order  to  accomplish  this  he  must  form  a  close  liaison  with 
the  other  design  disciplines.  The  "Design  Discipline  Testability  (T) 
Inter-Relationships"  section  of  each  compendium  is  a  suggested  list  of 
groups  that  the  testability  engineer  should  coordinate  with  for  that 
particular  task. 


2.3  Authorities  to  be  established  typically  include: 

-  Imposition  of  qualitative  and  quantitative  testability  requirements 
into  design  goals  and  into  Cl  specifications. 

-  Imposition  of  quantitative  testability  requirements  into  BIT  design 
and  ATE  or  ETE  selection. 


-  Imposition  of  testability  requirements  on  software  development. 


2.4  Responsibilities  to  be  accepted  by  testability  engineering  typically 

include : 

-  Timely  delivery  of  inputs  to  design  goals  and  Cl  specification 
documents . 

-  Timely  delivery  of  inputs  for  BIT  design  and  ATE  or  ETE  selection 
and  thorough  review  of  BIT  design  and  ATE  or  ETE  selection  tradeoffs. 

-  Timely  delivery  of  inputs  for  software  development  and  thorough 
review  of  program  lists  for  testability. 


2.5  The  Test  Program  Plan  will  be  contractually  required  by  CDRL/DID.  The 
degree  of  acceptability  can  be  measured  by: 

a.  Compliance  with  CDRL/DID  requirements. 

b.  Thoroughness  in  establishment  of  meaningful  assignments  of  authority 
and  responsibility  for  each  testability  task. 

c.  Thorough  documentation  of  organizational  interfaces. 

3.  Completion  Criteria. 

The  task  is  complete  upon  acceptance  by  the  government  of  the  Testability 
Program  Plan. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2A 


PHASE;  VALIDATION 

FUNCTION;  System  Design  T  Incorporation 


TASK  TITLE:  General  Testability  Design  Guidance 

TASK  OBJECTIVE:  This  task  is  somewhat  hypothetical  in  that  it  provides  a 

vehicle  for  listing  of  a  generalized  set  of  design  guide¬ 
lines  for  early  transmittal  to  the  designers. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System  Engineer¬ 
ing 

e  T  Design  Guidelines 

e  Safety 
Engineering 

e  T  Design  Guidelines 

e  Design 
Engineering 

e  T  Design  Guidelines 

#  Reliability 
Engineering 

e  T  Design  Guidelines 

e  Maintainability 
Engineering 

e  T  Design  Guidelines 

•  Logistics 
Engineering 

e  T  Design  Guidelines 

COST  TRADEOFF  INTER-RELATIONSHIPS;  Providing  early  guidance  to  the  designers 
minimizes  the  cost  of  design  by  avoiding  re-design  and/or  subsequent  engineer¬ 
ing  changes. 

TASK  SYNOPSIS; 

1.  Task  Requirements. 

This  task  is  a  vehicle  for  providing  early  guidance  of  a  general  nature  to  the 
designers.  It  is  aimed  primarily  at  circuit  card  level,  but  the  principles 
are  applicable  at  higher  levels  of  the  hardware  indenture.  Tasks  V2B  through 
V2I  provide  more  specialized  guidance  related  to  specific  aspects  of  test¬ 
ability  and  in  fact  repeat  some  parts  of  the  guidance  of  this  task.  The 
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guidelines  should  also  be  provided  for  information  to  engineering  disciplines 
other  than  design. 


2.  Implementation . 

The  appendix  hereto  is  a  separable  set  of  guidelines  which  may  be  directly 
provided  to  the  designer. 


3.  Completion  Criteria. 

The  task  is  complete  upon  completion  of  the  design  effort  with  all  realizable 
guidance  points  contained  in  the  design. 
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(2) 

APPENDIX  V2A1 

Testable  Design 

The  general  characteristics  of  a  testable  design  are: 
e  Test  control  of  internal  nodes,  including  initialization 
e  Observability  of  internal  nodes,  directly  or  inferable 
e  Mechanical  and  electrical  compatibility  with  available  ETE/ATE 
e  Functional  partitioning 
e  Conservative  timing  and  signal  tolerances 
e  Well-behaved  failure  modes 
e  Restricted  fan-out  count 
e  Simple,  straightforward,  regular  designs 

e  Support  of  testing  at  several  levels  of  hardware  indenture 


Test  Software  ^ 

e  Device  test  programs  should  be  programmed  in  an  ATE  high-order  source 
language,  e.g.,  ATLAS  or  OPAL. 

e  Source  programs  should  be  defined  in  independent  blocks  traceable  to  a 
functional  grouping  of  device  test  (UUT)  requirements. 

e  The  blocks  should  be  annotated  to  describe  their  test  functions. 

e  Block  size  should  be  restricted  to  test  sequence  size  (e.g.,  fifty  test 

measurements)  and  also  run-to-completion-time  (e.g.,  five  minutes). 

e  The  test  blocks  should  contain  a  common  set  of  language  procedures 
(subroutines  and  data  base)  wherever  possible. 

e  The  test  block  executions  should  be  individually  initiated  by  and 
terminated  into  the  test  executive  software. 
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PCB  Electrical  Layout 


•  Standardize  pin  location  of  common  signals  such  as  power ,  ground,  clock, 
clock  return  for  a  card  family.  Power  pins  should  be  located  on  the 
connector  ends  (board  edges) . 

e  Design  the  system  to  use  only  one  voltage  if  at  all  feasible  in  order  to 
avoid  potential  damage  that  may  result  from  transients  or  improper  voltage 
sequences. 

e  If  more  than  one  voltage  is  required,  segregate  multiple  voltages  to  as 
few  card  types  as  possible.  Provide  adequate  separation  to  prevent 
accidental  shorting  with  test  probes. 

e  Use  a  single  logic  family  for  the  logic  design.  If  more  than  one  logic 
family  must  be  used,  select  ones  with  common  power  requirements  and  pin 
assignments,  and  common  input/output  signal  compatibility  and  pin  assign¬ 
ments. 

e  If  logic  families  with  a  signal  interface  problem  must  be  used,  partition 
the  families  so  they  can  be  tested  individually. 

•  Select  logic  devices  that  are  independent  of  specific  clock  rates, 
controlled  rise  and  fall  times,  and/or  specific  gate  propagation  delays. 

•  Run  a  complete  characterization  test  study  on  new  type  devices  to  avoid 
sneaky  soft  (come  and  go)  problems.  Be  conservative  in  designs  that  use 
such  parts,  especially  with  respect  to  timing  parameters.  This  rule 
applies  with  strong  emphasis  to  complex,  large-scale  devices. 

•  Design  each  card  to  be  a  functionally  separate  package,  otherwise 
separate  multiple  functions  on  a  card  by  partitioning  so  that  each 
function  can  be  tested  separately. 

•  Subdivide  large  logic  boards  (  >100  ICs)  into  smaller  sections  for  easier 
testing.  Separate  the  Vcc  paths  or  use  tri-state  logic  to  allow  isolating 
one  section  from  the  others  for  testing. 

•  Do  not  mix  digital  and  analog  circuitry  on  the  same  card  if  feasible. 

This  does  not  include  simple  timing  circuits. 

e  If  both  digital  and  analog  circuitry  must  be  included  on  the  same  board, 
partition  the  board  into  digital  and  analog  portions  so  that  each  can  be 
tested  separately.  If  the  circuit  design  permits,  provide  the  digital/ 
analog  interconnections  externally  via  I/O  pins. 

e  Design  circuits  that  do  not  require  adjustments.  If  adjustments  must  be 
incorporated,  design  the  board  so  that  the  adjustment  can  be  made  and 
locked  during  test. 
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•  Design  I/O  lines  to  allow  for  inevitable  shorting  by  providing  current 
limiting  or  crowbar  protection  techniques. 

•  Design  I/O  lines  and  test  points  to  accommodate  capacitive  loads  larger 
than  intended  in  the  system.  This  enables  the  PCB  to  be  accommodated  by 
a  wider  range  of  test  adapters  and  ATE  sets. 

e  Provide  means  for  external  control  of  each  node  independent  of  the 
associated  logic  by  providing  direct  access  via  an  I/O  pin  or  a  test 
point.  This  allows  external  control  of  the  node  for  troubleshooting 
purposes. 

•  Provide  sufficient  access  to  the  internal  circuit  by  bringing  internal 
test  and  control  nodes  to  unused  pins  on  the  connector,  or  to  pins  on 
the  test  point  connector. 

e  Provide  simple  means  for  initializing  (setting  to  an  initial  state)  all 
storage  elements,  using  signals  applied  to  I/O  pins  or  test  points.  A 
single  direct  reset  is  preferred;  a  short  (<16  bit)  count  sequence  is 
acceptable. 

e  Make  display  refresh  circuitry  capable  of  being  disabled,  with  the 
refresh  cycle  then  being  capable  of  being  controlled  by  the  tester. 

e  Incorporate  built-in-test  capability  into  very  large  logic  boards  such 
that  BIT  can  be  initiated  by  the  ATE  and  the  test  results  can  be 
evaluated  by  the  ATE. 

e  If  access  to  internal  circuitry  of  a  PCB  is  limited,  provide  BIT 
capability  on  the  card  that  can  be  initiated  and  read  by  the  ATE. 

e  Use  display  LEDs  on  the  PCB  to  indicate  proper  operator  of  important 
circuits.  Examples  are:  power  supply  voltage  is  O.K. ,  clock  is 
present,  a  phase-locked  loop  is  locked,  etc. 

e  Use  position  indication  fault  indicators  and  displays  such  that  a  good 
test  always  results  in  an  "ON"  condition.  A  defective  indicator  or 
display  then  always  indicates  a  fault  condition,  either  due  to  the 
input  or  of  itself  (if  it  is  faulty). 

For  a  critical  display,  provide  an  alternate  method  of  testing  such 
as  "push-to-test"  to  provide  positive  verification  of  its  operation. 
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Current  Requirements  and  Limitations 

e  Limit  input  signal  requirements  including  clock  and  master  clear  to  <20ma 
Provide  on-board  buffering  if  more  drive  capability  is  required. 


e  Buffer  clock  inputs  before  they  fan  out  to  avoid  loading  down  clock  drive 
outputs . 

e  Provide  sufficient  current  capability  in  high  frequency  and  pulse  outputs 
to  drive  a  low  impedance  transmission  line. 


e  Provide  current  limiting  on  output  lines  where  there  is  a  likelihood  of  a 
cascading  of  failures,  should  a  single  component  fail. 


e  Do  not  tie  direct  set  or  reset  lines  to  Vcc  or  ground.  Decouple  Vcc 

through  a  resistor  to  provide  a  logic  high;  pass  a  decoupled  high  through 
an  inverter  to  provide  a  logic  low.  This  protects  against  shorting  Vcc 
to  ground  due  to  a  component  failure.  (See  figure  1.) 

e  Alternatively  use  a  grounded  input  inverter  to  provide  a  logic  high,  and 
a  second  inverter  driven  by  the  first  to  provide  a  logic  low.  This 
greatly  aids  in  locating  a  fault  due  to  a  shorted  fixed  "zero"  or  fixed 
"one"  input. 


e  In  some  cases  where  real  estate  is  a  problem,  a  logic  signal  with  a  diode 
input  can  be  safely  tied  to  Vcc. 


V2A1-4 


Design  I/O  lines  to  allow  for  inevitable  shorting  by  providing  current 
limiting  or  crowbar  protection  techniques. 

Design  I/O  lines  and  test  points  to  accommodate  capacitive  loads  larger 
than  intended  in  the  system.  This  enables  the  PCB  to  be  accommodated  by 
a  wider  range  of  test  adapters  and  ATE  sets. 

Provide  means  for  external  control  of  each  node  independent  of  the 
associated  logic  by  providing  direct  access  via  an  I/O  pin  or  a  test 
point.  This  allows  external  control  of  the  node  for  troubleshooting 
purposes. 

Provide  sufficient  access  to  the  internal  circuit  by  bringing  internal 
test  and  control  nodes  to  unused  pins  on  the  connector,  or  to  pins  on 
the  test  point  connector. 

Provide  simple  means  for  initializing  (setting  to  an  initial  state)  all 
storage  elements,  using  signals  applied  to  I/O  pins  or  test  points,  A 
single  direct  reset  is  preferred;  a  short  (<16  bit)  count  sequence  is 
acceptable . 

Make  display  refresh  circuitry  capable  of  being  disabled,  with  the 
refresh  cycle  then  being  capable  of  being  controlled  by  the  tester. 

Incorporate  built-in-test  capability  into  very  large  logic  boards  such 
that  BIT  can  be  initiated  by  the  ATE  and  the  test  results  can  be 
evaluated  by  the  ATE. 

If  access  to  internal  circuitry  of  a  PCB  is  limited,  provide  BIT 
capability  on  the  card  that  can  be  initiated  and  read  by  the  ATE. 

Use  display  LEDs  on  the  PCB  to  indicate  proper  operator  of  important 
circuits.  Examples  are;  power  supply  voltage  is  O.K. ,  clock  is 
present,  a  phase-locked  loop  is  locked,  etc. 

Use  position  indication  fault  indicators  and  displays  such  that  a  good 
test  always  results  in  an  "ON"  condition.  A  defective  indicator  or 
display  then  always  indicates  a  fault  condition,  either  due  to  the 
input  or  of  itself  (if  it  is  faulty) . 

For  a  critical  display,  provide  an  alternate  method  of  testing  such 
as  "push-to-test"  to  provide  positive  verification  of  its  operation. 
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Current  Requirements  and  Liwl-tatlons 

e  Limit  input  signal  requirements"tfKJ.uding  clock  and  master  clear  to  <20ma. 
Provide  on-board  buffering  if  store  drive., capability  is  required. 

e  Buffer  clock  inputs  before  they  fan  out  to  avoid"lcading  down  clock  drive 
outputs. 


Provide  sufficient  current  capability  in  high  frequency  and  pulie^qutputs 
to  drive  a  low  impedance  transmission  line. 


e  Provide  current  limiting  on  output  lines  where  there  is  a  likelihood  of  a 
cascading  of  failures,  should  a  single  component  fail. 


e  Do  not  tie  direct  set  or  reset  lines  to  Vcc  or  ground.  Decouple  Vcc 

through  a  resistor  to  provide  a  logic  high;  pass  a  decoupled  high  through 
an  inverter  to  provide  a  logic  low.  This  protects  against  shorting  Vcc 
to  ground  due  to  a  component  failure.  (See  figure  1.) 


e  Alternatively  use  a  grounded  input  inverter  to  provide  a  logic  high,  and 
a  second  inverter  driven  by  the  first  to  provide  a  logic  low.  This 
greatly  aids  in  locating  a  fault  due  to  a  shorted  fixed  "zero"  or  fixed 
"one"  input. 


Figure  1.  Isolate  Logic  Highs  and  Lows  From  Power  Buses 


e  In  some  cases  where  real  estate  is  a  problem,  a  logic  signal  with  a  diode 
input  can  be  safely  tied  to  Vcc. 
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Redundant  Circuitry 

e  Avoid  logically  redundant  circuits.  A  connection  in  a  circuit  is  said  to 
be  redundant  if  no  change  occurs  in  the  output  of  the  circuit  when  the 
connection  is  open.  (See  figure  2) 


Figure  2.  Provide  Test  Access  by  Redesign  to  Avoid  Fault  Redundancy 


Both  circuits  generate  the  same  logic  function.  However  circuit  (a) 
gives  the  same  output  even  when  the  B  input  on  gate  Gl  is  stuck  at  1, 
or  the  C  input  on  gate  G2  is  stuck  at  1.  In  either  case  the  output 
becomes  F  =*  ABC+AD ,  hence  the  fault  cannot  be  isolated  to  a  specific 
gate. 

With  either  B  or  C  stuck  at  one,  circuit  (b)  output  also  becomes 
F  =  ABC+AD,  but  now  the  fault  can  be  isolated  to  the  B'C  gate. 


e  Logically  redundant  circuits  can  help  solve  race  problems  by  providing 
overlapping  control  at  a  clock  time. 


e  Redundant  design  cam  prevent  a  glitch.  (See  figure  3) 
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Figure  3.  Gate  G3  is  Redundant,  as  can  be  seen  on  the  Veitch  Diagram 


Without  G3,  a  glitch  could  occur  at  F  during  the  crossover  time 
when  gate  G2  (A*C)  releases  control  and  gate  Gl  (A*B)  takes  control 
(or  vice  versa).  The  glitch  could  direct  set  the  flip-flop.  By  adding 
the  redundant  gate  G3  (B'C),  the  glitch  cannot  occur. 


Wired  AND  and  Wired  OR  Connections 


•  Do  not  use  wired  AND  or  OR  because  of  the  ambiguity  that  is  created  in 
trying  to  localize  a  fault  to  the  specific  faulty  gate. 


•  Do  not  use  wired  AND  (OR)  with  high  powered  TTL  or  Schottky  TTL;  they 
cannot  be  direct  driven  low  safely  for  more  than  1  second. 

•  Break  wired  AND  connections  into  smaller  ambiguity  groups  as  illustrated. 
(See  figure  4)  The  before  case  shows  six  gates  were  ANDed;  the  after 
case  shows  three  pairs  of  wired  ANDed  gates.  The  ambiguity  factor  has 
been  reduced  from  6  to  3. 
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Figure  4.  Redesign  Gate  Networks  to  Reduce  Fault  Ambiguity  Factor 


Use  wired  AND  (OR)  for  external  control  connections  as  a  means  of 
injecting  test  stimuli  into  card  under  test.  (See  figure  5.) 


Figure  5.  Wire  OR  for  Test  Control  Input 


I 


V2A1-7 


Internal  Clocks  and  Counter  Chains 


•  Use  synchronous  clocking  systems  only.  This  reduces  potential  timing 
hazards  to  narrow  and  easily  identifiable  areas  of  logic.  With  syn¬ 
chronous  clocking  the  typical  digital  logic  assembly  can  be  tested 
on  a  static  digital  test  system  with  less  concern  about  timing  response 


e  Provide  an  easy  means  of  disabling  or  bypassing  on-board  clocks 

(oscillators,  etc.)  so  that  the  necessary  clock  stimuli  can  be  provided 
from  the  test  equipment.  (See  figure  6.) 
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Figure  6.  Techniques  for  Disabling  On-Board  Clocks  to  Allow  Off-Board  Clocking 


e  Provide  means  for  easily  inhibiting  an  on-board  oscillator  (putting  it 
into  a  non-oscillatory  mode). 

e  Break  up  long  counter  chains  into  smaller  segments,  preferably  of  equal 
length,  such  that  the  total  count  time  provides  a  reasonable  total  test 
time.  (See  figures  7,  8  and  9.) 


UNOIVfOf  DCOUNTI A  CHAIN.  TSST  TIMS  lti">T 


Figure  7. 


IMPHOVIO  CUUN I  liH  CHAIN.  TtST  TIM!  I*  j",> 


Figure  8. 
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Figure  9. 
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Sequential  Circuits 

e  Initialize  all  sequential  circuits  to  a  known  start  condition  utilizing 
the  shortest  possible  sequence,  ideally  one  transition,  and  never  more 
than  20  transitions. 

e  Provide  means  to  initialize  all  sequential  logic  elements  from  I/O 

pins  or  test  points.  This  applies  to  flip-flops,  counters,  registers, 
memories,  etc.  (See  figure  10.) 


Figure  10.  Flip-Flop  Modified  for  External  Reset  Control 


e  If  an  I/O  pin  or  test  point  is  not  available,  provide  initialization  by 
means  of  "power-up."  (See  figure  11.) 


e  If  unused  direct  reset  and  set  pins  must  be  tied  down,  always  do  so 
through  a  decoupling  resistor.  This  can  make  these  inputs  available 
for  test  purposes,  and  prevents  shorting  the  power  bus  through  a  bad 
chip. 
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•  Avoid  arbitrary  use  of  edge-triggered  devices  such  as  D  flip-flops  and 
J-K  flip-flops.  They  increase  modeling  requirements,  yield  lower  test 
quality  in  nodal-fault  testing,  and  often  exhibit  race  conditions. 

e  Provide  access  for  test  control  to  internal  nodes  of  complex  sequential 
circuits  by  means  of  gates,  X/O  pins,  and/or  test  points.  (See 
figure  12.) 
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Figure  12.  Provide  Access  to  Internal  Modes  for  Test  Control 


•  Provide  positive  controls  to  prevent  logic  lockups  that  can  be  cleared 
only  by  a  master  clear,  or  indeterminate  outputs  such  as  can  occur  from 
an  R/S  flip-flop. 


e  Logic  designers  should  provide  state  diagrams  for  sequential  logic  cir¬ 
cuits.  This  will  enable  the  test  engineer  to  analyse  tests  from  valid 
sequences  only,  and  to  ensure  that  initialization  prevents  the  occurrence 
of  invalid  sequences  and  lockup. 
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Asynchronous  Circuits,  Monostables  (One-Shots)  and  Feedback  Networks 


Do  not  use  asynchronous  clocking.  It  typically  requires  use  of  edge- 
triggered  flip-flops,  nonostables  (one-shots)  and/or  delay  elements  to 
control  the  functional  sequence.  Testing  monostables  and  delay  lines 
usually  requires  manual  test  generation  for  accurate  timing  tests, 
voiding  the  use  of  automatic  test  generation. 

Asynchronous  clocking  is  highly  undesirable  because  it  usually  results 
in  the  use  of  various  asynchronous  clocking  methods  distributed  through¬ 
out  a  functional  module.  This,  in  turn,  establishes  a  need  for  a 
multitude  of  separate  test  environments  and  extensive  costly  manual 
test  generation. 

Do  not  use  asynchronous  circuits.  If  they  must  be  used,  provide  means 
for  synchronizing  them  in  test,  or  provide  means  so  that  they  can  be 
tested  by  themselves. 

Do  not  use  monostables  on  digital  logic  boards. 

If  monostables  must  be  used,  they  must  be  capable  of  producing  synchronized 
predictable,  and  repeatable  pulses. 
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Figure  13.  Electrical  Disconnection  of  Monostables  with  External  Simulation  Path  Provided 
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Large  Scale  Rules 

The  large  scale  testability  design  rules  are  those  that  apply  particularly 
to  LSI  and  VLSI  devices  including  microprocessors,  microprocessor  support 
chips,  RAMs  and  ROMs,  UARTs,  etc. 

Some  of  the  rules  already  discussed  apply  in  general  to  large  scale  devices, 
when  appropriately  modified  to  meet  the  special  conditions  applicable  to  such 
devices.  The  rules  below  are  those  particularly  applicable  to  design  using 
large  scale  devices. 

e  The  card  designer  and  the  test  engineer  must  both  have  detailed  knowledge 
of  large  scale  components  and  the  problems  of  interfacing  between  them. 
This  knowledge  is  required  to  develop  a  viable  design  that  includes  the 
testability  needed  to  detect  faulty  operation  and  to  isolate  the  fault 
to  a  defective  component. 

•  Test  each  lot  of  an  LSI  device  to  ensure  that  the  device  characteristics 
have  not  changed.  A  different  lot  number  of  a  large  scale  device, 
particularly  in  its  early  life,  can  mean  differences  in  characteristics. 

e  Characterization  studies  to  establish  worst  case  limits  of  critical 
parameters  are  essential  for  LSI  devices.  Semiconductor  manufacturers 
generally  do  not  provide  information  on  test  vector  patterns  that  best 
test  their  devices:  They  generally  only  furnish  DC  and  timing  charac¬ 
teristics.  User  experience  is  that  characterization  studies  are  worth 
their  weight  in  gold. 

•  Use  dynamic  devices  (dynamic  RAM,  microprocessors,  etc.)  only  when 
absolutely  required.  Provide  separate  access  to  the  refresh  clock  at 
an  I/O  pin  or  test  point. 

e  The  initialization  rules  also  apply  specifically  to  volatile  memory 
elements.  Conditional  initialization  iB  not  acceptable  without  a 
controlling  override  absolute  initialization.  A  predetermined  short 
sequence  of  clock  counts  to  initialize  is  acceptable. 

e  Use  partitioning  judiciously  to  separate  the  functioning  elements  of 
a  product  to  improve  overall  testability  characteristics. 

e  Avoid  concentrating  many  LSI  chips  on  a  single  PC  board.  The  resulting 
structure  is  difficult  to  test  and  troubleshoot  because  of  the  restricted 
physical  access  and  the  restricted  test  visibility. 

•  Utilize  chips  with  power  ON/OFF  capability  to  electrically  partition 
groups  of  chips  for  independent  testing  of  parts  of  the  PCB  one  at  a 
time.  Then  the  parts  can  be  combined  piecewise  or  en  masse,  depending 
upon  interface  complexity.  This  approach  greatly  simplifies  fault 
localization  and  applies  the  proven  concept  of  bottom-up  integration 
PCB  testing. 
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•  After  checking  specification  requirements ,  if  possible,  mount  all  large 
and/or  complex  ICs  in  sockets  so  that  they  can  be  removed  to  allow 
testing  by  partitioning. 

e  When. doing  characterization  studies  of  an  LSI  device,  know  what  phase  in 
the  life  cycle  the  device  has  achieved.  These  phases  are  as  follows: 

Characterization  Phase-tests  define  the  critical  characteristics 
and  characteristic  limits.  Processes  and  parameters  are  subject 
to  change  as  the  device  matures.  Production  is  low. 

Prematurity  Phase-  production  is  increasing  and  processes  and 
parameters  are  more  stabilized. 

Shell  Phase-  production  growth  is  rapid.  Tests  are  pared  to  a 
"shell"  of  critical  requirements  and  some  testing  of  critical 
processes.  Dedicated  test  equipment  is  selected.  Competition  in 
production  is  apparent. 

Maturity  Phase-  production  has  peaked.  Dedicated  test  equipment 
is  used  for  stationized  testing  of  the  "shell”  of  tests  selected  in 
the  previous  phase.  Process  control  testing  is  minimal  but  adequate. 
Competition  is  reaching  its  peak. 

Post  Maturity  Phase-  the  device  is  turned  over  to  the  sustaining 
production  group.  Routine  testing,  based  on  past  learning,  is 
done  at  minimal  cost  but  with  tight  control  of  processes.  Prices 
are  fixed.  Competition  has  decreased  but  also  has  stabilized. 


Microprocessor  and  Support  Chip  Rules 

•  Capture  the  microprocessor  design  "tfort  for  later  analysis  and  use  in 
the  test  development  effort.  Inc  iae  all  data  derived  from  the  use  of 
design  and  debugging  aids  such  as  an  MDA  (microprocessor  development 
aid) ,  logic  analyzer  and/or  other  design  tools,  including  complete 
information  on  all  equipment  setups,  procedures,  and  results.  Use 
computerized  tools  to  perform  this  data  capture  automatically  and 
painlessly. 

•  In  a  microprocessor  system  the  functional  operating  characteristics  of 
the  circuit  are  not  necessarily  associated  with  specific  hardware 
components. 

e  Microprocessor  systems  generally  lend  themselves  well  to  ordered  par¬ 
titioning.  The  average  microprocessor  system  has  six  unique  chips. 

e  Due  to  their  complexity,  microprocessors  are  very  difficult  to  test  when 
incorporated  into  a  PCB  as  part  of  the  logic  design.  If  they  can  be 
mounted  in  sockets,  this  permits  removing  them  for  testing  separate  from 
the  rest  of  the  PCB. 
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•  Use  microprocessors  from  the  same  family  in  a  multiprocessor  configura¬ 
tion,  rather  them  different  microprocessors  unless  there  are  specific 
compelling  reasons  to  do  so.  Multiprocessor  configurations  developed 
around  a  microprocessor  family  are  much  easier  to  test. 

e  Verify  that  the  microprocessor  can  be  run  in  the  tri-state  mode  (high 
impedance  state) . 

•  Provide  inhibit  gates  for  microprocessor  outputs  that  cannot  be  put  into 
the  tri-state  high  impedance  state. 

e  Separate  bit-slice  microprocessors  into  elementary  slices  for  test  control. 
This  permits  the  use  of  a  much  simpler  program  to  test  each  elementary 
slice  independently.  Inter slice  coupling  can  then  be  tested  by  a  rela¬ 
tively  simple  test  program. 

•  Tie  unused  CPU  control  inputs  to  power  buses  only  through  resistors  or 
inverters,  and  provide  access  through  I/O  pins  or  test  points.  This 
provides  access  to  the  control  line  for  test  purposes  and  prevents 
shorting  of  the  power  bus  if  the  device  is  defective. 

•  Microprocessor  phase  clocks  must  conform  strictly  to  the  requirements  of 
the  microprocessor  and  its  associated  ships,  xather  than  being  dependent 
upon  any  other  clock  circuitry. 

e  Provide  a  means  to  loop  I/O  port  outputs  back  to  inputs  for  a  very 
effective  I/O  test.  This  has  been  implemented  using  an  on-board  test 
connector  together  with  a  test  adaptor  connector  at  the  board  edge. 

•  Test  the  clock  circuit  immediately  after  POWER  UP  tests.  An  absent, 
jittery,  or  noisy  clock  can  play  havoc  with  the  rest  of  the  hardware 
functions.  Abberations  of  the  clock  signal  may  prove  fatal,  and 
marginal  operation  often  leads  to  hard-to-find  transient  problems. 

•  Use  progressive  testing  to  verify  the  hardware ,  employing  simple  programs. 
The  best  test  sequence  is  Power,  Clock,  ROM,  RAM,  through  test  of  I/O 
ports  and  interrupts,  and  bus  control  logic. 

•  Test  non-CPU  circuitry  independently  by  tri-stating  the  CPU  and 
utilizing  the  inhibit  gates  to  isolate  the  CPU  from  the  other  circuitry. 

•  Perform  passive  preliminary  tests  to  be  sure  that  the  hoard  is  free 
for  communication. 

•  The  three  widely  used  microprocessor  test  methods  are: 

1.  Actual  use 

2.  Stored  response  from  a  known  good  board 

3.  Algorithmic  pattern  generation 
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•  Subdivide  the  microprocessor  into  functional  units  (adder,  multipier, 
incrementer/decrementer ,  program  counter,  input  buffer,  output  buffer, 
stack  pointer,  register  address,  etc.).  Devise  a  diagnostic  fault 
detection  only  test  for  each  of  these  units,  since  they  cannot  be 
repaired.  Utilize  the  principle  of  "expanding  on  the  kernel"  in 
devising  the  sequence  of  these  tests. 

e  The  principles  of  individual  functional  subunit  testing  and  expanding- 
on-the-kernel  provides  a  controlled  progressive  means  of  developing 
programs.  It  keeps  one  from  being  overwhelmed  by  the  complexity  of 
the  device. 

e  Devise  a  test  for  the  microprocessor  kernel,  the  minimum  configuration 
of  microprocessor  and  ROM  needed  to  run  a  very  simple  test  program.  Use 
this  proven  part  to  expand  testing  to  an  untested  area.  Continue  this 
test  expansion  process  until  the  complete  microcomputer  system  has  been 
tested. 

e  Properly  designed-in  testability  can  significantly  simplify  testing  of 
microcomputers.  Three  reasons  support  this  conclusion:  (1)  resident 
self -test  programs  written  in  the  microcomputer's  own  language  can  be 
stored  directly  on  ROM.  By  "expanding  on  the  kernel"  these  programs 
provide  excellent  means  for  verification  of  operation,  and  can  easily 
generate  ample  stimulus  for  fault  diagnosis.  The  same  self-test  can  be 
used  for  proving  out  the  design  in  the  laboratory,  testing  in  production 
and  servicing  in  the  field;  (2)  the  system  bus,  which  is  centrally 
located  and  connects  to  many  components  in  the  system,  can  be  physically 
designed  to  provide  easy  access  for  application  of  test  stimuli  and 
visibility  of  test  results;  (3)  bus  oriented  architectures  are  inherently 
readily  designed  into  easily  diagnosable  modules. 

e  All  software  storage  locations  that  are  accessed  by  the  CPU  must  be 
initialized  by  an  initializing  routine  that  precedes  the  test  routine 
proper.  Include  in  the  initializing  sequence  RAM  locations,  peripheral 
IC  registers,  input  ports,  CPU  registers  and  PIDs,  etc.,  that  are  not 
affected  by  hardware  initialization.  The  test  designer  must  be  aware 
of  exactly  what  registers  and  addresses  are  saved  on  the  stack  at  the 
initiation  of  an  interrupt  or  subroutine  call.  When  the  return  is 
executed  these  saved  registers  are  read  back  from  the  stack  into  the 
processor  locations,  etc.,  from  which  they  were  saved.  The  initializa¬ 
tion  procedure  must  precede  any  interrupt  action  or  subroutine  call  to 
ensure  that  proper  control  of  these  registers,  etc.,  is  maintained  at 
all  times. 

e  Microcomputer  design  should  include  a  built-in  self-test,  which  requires 
about  1  K-byte  of  ROM.  A  software  sequence,  external  switch,  or  test 
point  can  be  used  for  the  initiating  maintenance  action  of  this  test. 

e  An  alternative  to  built-in  test  is  to  utilize  an  external  signal,  switch- 
action,  or  connector  to  overlay  a  self-test  program  onto  the  applications 
program  RAM  memory  space,  and  operate  the  test  from  there. 
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•  Write  general  purpose  self-test  nodules  that  apply  to  any  system  based 
on  a  particular  microprocessor;  for  example,  a  CPU  test  module  and  a 
RAM  test  module.  Usually  only  small  changes  are  required  to  adapt  the 
test  modules  from  one  system  to  another. 

e  Write  I/O  driver  routines  in  applications  programs  as  subroutines  so 
that  they  can  also  be  incorporated  into  test  routines  as  a  subroutine 
call.  This  saves  both  code  space  and  program  development  time. 

e  The  first  step  to  localize  a  microprocessor  system  r..alfunction  is  to 
determine  its  generic  source.  Is  it  caused  by  hardware,  firmware  or 
software? 


Buses 

e  The  first  step  in  functional  board  test  of  a  microcomputer  board  is  to 
make  sure  the  bus  structure  is  free  of  manufacturing  faults.  This 
should  be  done  even  when  the  board  has  been  through  in-circuit  component 
inspection  test  (ICCIT) . 

e  Provide  access  to  microcomputer,  RAM,  ROM  (or  other  device)  address, 

data,  and  control  buses  as  applicable.  This  can  be  done  in  a  number  of 
ways. 


The  most  flexible  access  is  to  bring  all  buses  to  I/O  connector  pins. 

Route  buses  that  do  not  have  I/O  access  to  test  points  (a  test 
connector) . 

Provide  space  to  access  the  buses  by  means  of  a  dip-clip.  (If  the 
boards  are  to  be  conformally  coated  this  is  not  useful  for  post¬ 
delivery  factory  retest  or  for  field  testing) . 

Provide  access  to  the  control  line  for  tri-state  buses  at  I/O  pins 
or  test  points. 


e  The  rule  for^ putting  pull-up  resistors  on  the  same  board  as  their  driving 
device  applies  particularly  to  tri-state  devices. 

e  Add  a  default  bus  driver  to  actively  pull  the  bus  to  a  known*  state  when 
no  other  bus  drive  is  active.  Decode  hardware,  using  address  and  bus 
control  inputs,  is  used  to  generate  the  default  bus  control. 

e  Provide  a  means  for  floating  or  opening  each  bus  line  to  assist  in  fault 
isolation  and  to  follow  free  running  operations.  Use  one  or  a  combina¬ 
tion  of  the  following  methods; 

-  Hardware  or  software  controlled  chip  select  lines,  jumper  wires, 
shorting  plugs,  socketed  components. 
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In  multiple  board  systems r  provide  a  separate  extender  board  with 
a  Switch  on  each  data  lint.  Use  this  board  to  isolate  the  suspected 
board  from  the  others.  By  closing  one  switch  at  a  time  and  observ¬ 
ing  results,  a  specific  bad  bus  can  be  detected. 

With  all  data  bus  switches  on  the  extender  open  and  address  bus 
switches  closed,  a  failure  can  be  localized  without  the  rush  of  bad 
data  feedback  that  can  invalidate  the  control  test  stimulus  patterns. 
Electronic  switching  will  allow  the  ATE  to  control  the  testing  and 
fault  detection  process. 


e  Use  a  current  tracing  tool  to  probe  beyond  the  node  level.  Models  are 
available  that  can  be  used  off-line  to  trace  the  current  on  a  faulty  bus 
line  to  the  source  device. 

e  Do  not  use  multilayer  boards  where  there  are  bus  lines  coupling  several 
devices.  It  is  difficult  to  use  a  current  tracer  on  such  a  board  to 
discover  which  device  is  holding  a  bus  line  high  or  low. 


RAMs,  ROMs,  Serial  Registers 

•  Pattern  sensitivity  studies  must  be  made  for  all  large  scale  memory 

devices.  Some  current  memories  use  as  little  as  28%  of  the  real  estate 
for  the  actual  memory  area.  Circuit  features  other  than  the  memory  array 
are  now  primarily  responsible  for  pattern  sensitivity. 

e  Isolate  large  memory  devices  (1024  bits  or  larger)  from  significant 
amounts  of  standard  logic.  Consider  the  requirement  of  special  test 
techniques  and  equipment  for  testing  large  memory. 

e  Provide  access  to  control  (enable)  lines  and  output  lines  of  memory 

devices  at  I/O  pins  or  test  points.  If  this  is  not  possible,  the  memory 
devices  should  be  mounted  in  sockets  so  that  they  can  be  removed.  This 
permits  a  simpler  test  program  for  the  PCB  and  a  separate  test  for  the 
memory  device. 

e  Provide  means  in  hardware  or  software  to  isolate  ROMs  from  each  other 
and  from  other  bus  elements.  This  can  include  providing  ROM  sockets  to 
permit  physical  removal.  This  will  allow  isolation  of  address  decode 
faults  that  result  in  incorrect  data  being  read  onto  the  bus. 

e  Provide  a  known  PROM  output  state  for  every  input  address  combination. 

e  Define  any  unused  or  don't  care  ROM  states  such  that  accidental  entry 
into  these  states  will  bring  the  system  back  to  a  defined  state. 

e  Provide  means  to  disable,  isolate,  or  remove  each  ROM  individually  in 
order  to  isolate  the  ROMs  from  each  other  and  from  other  bus  elements. 

The  means  can  use  hardware  or  software  techniques,  or  a  combination. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2B 


PHASE; 

FUNCTION: 

TASK  TITLE; 


VALIDATION 

System  Design,  Testability  Incorporation 

Provide  for  design  compatibility  of  units  to  be  tested 
with  selected  or  available  ETE  or  ATE. 


TASK  OBJECTIVES;  1.  Discover  and  resolve  any  test  interface  inonpati- 

bilities  between  the  design  and  the  ETE  or  ATE. 

2.  Reduce  or  eliminate  the  need  for  a  large  number  of 
unique  Interface  Device  (ID)  designs,  and  identify 
and  specify  the  requirements  for  any  unique  ETE,  ATE 
or  IDs. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 

e  System  and  UUT  Performance 

e  T  Design  Guidelines 

Engineering 

Data 

e  Design 

e  UUT  Interface  Data 

e  ID  Related  Requirements, 

Engineering 

Interface  Incompatibility 

Resolutions 

e  Logistics 

e  ETE/ATE  Alternatives 

e  Interface  Incompatibility 

Engineering 

Resolutions 

e  Test  Equipment 

e  ETE/ATE  Interface  Data 

•  Interface  Incompatibility 

Engineering 

Resolutions 

COST  TRADEOFF  INTER-RELATIONSHIPS;  Costs  of  development,  acquisition  and 
support  are  in  general  directly  proportional  to  the  diversity  of  test  equip¬ 
ments  and  devices  introduced  with  the  prime  system.  Therefore  new  test 
equipments  and  the  range  and  depth  of  interface  devices  and  software  should 
be  kept  to  lowest  possible  minima. 


TASK  SYNOPSIS: 


1.  Task  Requirements . 

1.1  ETE/ATE  Compatibility  Verification  Process 

-  ATE/UUT*  system  compatibility  can  be  determined  by  consideration  of  the 
following  areas: 

a.  UUT  Packaging 

b.  Physical  interface  between  UUT  and  proposed  ETE  or  ATE 

c.  Electronic  and  power  interface  between  UUT  and  proposed  ETE  or  ATE 

1.2  specific  Characteristics 

-  The  following  specific  characteristics  and  features  should  be  analyzed: 

a.  Functional  packaging  scheme  that  results  in  UUTs  that  can  be 
tested  independently  and  singly 

b.  Test  point  access  and  pin  placement  to  permit  testing  to  lowest 
level  required  and/or  necessary  for  validity 

c.  System  and  UUT  design  amenable  to  tast  by  proposed  ETE  or  ATE  test 
stimuli  and  measurement  devices.  This  will  include  accuracies 
required  and  available,  accuracy  ratio  calculations,  resource 
matching,  and  timing  requirements. 

d.  Uniform  and  compatible  interface  plugs  and  receptacles 
«.  Power  source  compatibility 

f.  Provision  (to  extent  necessary)  for  manual  intervention  to  allow 
operator  to  make  adjustments  to  variable  components 

g.  Minimized  need  for  equipment  external  to  ETE  or  ATE  to  generate 
signals  or  monitor  responses 

2.  Implementation . 

2.1  In  all  interfaces,  the  testability  engineer  should  be  prompt,  active  and 
assertive.  Testability  is  most  effectively  instilled  into  a  system  if 
design  guidelines  are  provided  when  design  commences.  It  is  inefficient 
for  the  testability  (or  any  other  -ility)  engineer  to  take  the  role  of  a 
post-design  critic  and  difficult  as  well  as  uneconomical  for  designers 
to  incorporate  changes  to  documented  designs  when  those  changes  reflect 
design  criteria  which  could  have  and  should  have  been  disclosed  during 
or  ahead  of  design  formulation.  Timeliness  is  therefore  a  principal 
responsibility  for  the  testability  engineer. 


*  The  term  Unit  Under  Test  (UUT)  is  used  in  a  general  sense  and  may  be 
construed  equivalently  to  LRU,  SRU,  WRA,  SRA,  Module  or  any  accepted 
term  for  a  testable  component. 
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2.2  The  following  checklists  provide  guidance  for  the  compatibility  task. 

The  ETE  and  ATE  Compatibility  Checklist  includes  a  numerical  scoring 
feature  for  use  in  conjunction  with  the  Completion  Criteria  of  the  Task 
Synopsis. 

The  checklists  treat  hierarchial  indenture  levels  of  prime  systems.  For 
clarity,  two  levels  are  illustrated  as  successively  lower  than  the  "LRU” 
le-v  al  for  Which  test  initiation  (in  the  context  of  this  synopsis)  is 
assumed.  Definitions  for  LRU,  SRU  and  sub-SRU  as  follows  apply  to  terms 
used  in  the  checklist. 

LRU  is  a  generic  term  that  may  be  defined  in  terms  of  both  Avionic 
equipment  and  Ground  Systems  equipment. 

-  For  Avirnic  equipment  or  systems,  it  is  Line  Replaceable  Unit. 
Ground  Electronics  equipment  it  is  Lowest  Replaceable  Unit. 

LRU  is  defined  as  a  unit  which  is  designated  by  the  plan  for  main¬ 
tenance  to  be  removed  upon  failure  from  a  larger  entity  (equipment, 
system)  in  the  latter's  operational  environment.  (MIL-STD-1309-B) 

A  LRU  is  composed  entirely  of  SRU's.  A  SRU  is  defined  as  follows: 

SRU  (Shop  Replaceable  Unit)  -  a  generic  term  which  includes  all  the 
packages  within  a  LRU,  which  may  include  circuit  boards,  chassis, 
wiring  amasses  and  piece  parts  removed  at  the  shop  (intermediate 
or  depot)  level. 

3ub-SRU  -  a  generic  term  referring  to  a  smaller  circuit  board  or 
other  device  comprised  of  two  or  more  piece  parts  mounted  on  a  SRU. 

ETE  and  ATE  Compatibility  Checklist  -  Candidate  Quantitative  Evaluation 
Functional  Modularity. 

Determine  if  the  LRU  is  functionally  modularized  at  all  levels  of 
a8sembly/disassembl y. 

a.  Determine :  Score 

(1)  Each  LRU  function  is  contained  within  a  single  4 

SRU  and  each  SRU  function  is  contained  within  a 
Sub-SPU. 

(2)  t:  e  LRU  is  functionally  modularized,  but  some  SRU  3 

functions  are  not  modularized  within  Sub-SRUs. 
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Score 


(3)  A  few  LRU  functions  are  contained  cm  more  than  2 

one  SRU  and/or  most  SRUs  are  not  functionally 
modularized. 

(4)  Most  LRU  functions  encompass  more  than  one  SRU.  0 


Functional  Independence.  Determine  if  the  LRU  and  its  SRUs  are 
capable  of  being  tested  without  stimulation  by  another  LRU  or  SRU  and 


without  simulation  of  another  LRU  or  SRU. 

a.  Determine :  Score 

(1)  The  LRU  and  all  SRUs  are  functionally  independent.  4 

(2)  Some  SRUs  require  simulation  within  the  ID  using  2 

passive  and/or  simple  active  elements. 

(3)  Stimulation  by  another  LRU  or  SRU  is  required,  or  0 

complex  simulation  is  required. 


Adjustments.  Determine  if  adjustments  (e.g.,  trimming,  tuning,  align¬ 
ment)  must  be  made  while  testing  on  ETE/ATE .  An  adjustment  includes  any 
action  that  changes  variable  components  such  as  potentiometers ,  variable 
capacitors,  inductors,  transformers,  etc.,  that  affect  operation  of  the 


equipment. 

a.  Determine:  Score 

(1)  No  adjustments  or  realignments  are  necessary  for  4 

the  LRU  and  its  SRUs. 

(2)  A  small  number  of  simple  non-interactive  adjustments  3 
are  required,  but  no  complex  adjustment  or  realign¬ 
ment  is  required. 

(3)  One  or  two  SRUs  require  complex  adjustment  or  2 

realignment. 

(4)  The  LRU  or  more  than  two  SRUs  require  complex  0 

adjustment  or  alignment. 


External  Teat  Eguigment.  Determine  whether  external  equipment  is 
required  to  generate  a  stimulus  or  to  monitor  response  signals. 


a.  Determine :  Score 

(1)  All  stimulus  generation  and  response  monitoring  can  4 
be  accomplished  by  the  target  ETE/ATE. 

(2)  Signal  generation ,  synchronization,  or  waveshaping  2 

circuits  are  required  within  the  ID. 

(3)  Additional  external  test  equipment  is  required.  0 


Environmental .  Determine  if  the  LRU  or  the  SRUs  require  special 

environmental  considerations  during  test  on  the  ETE/ATE,  such  as  vacuum 
chambers,  oil  baths,  shake  tables,  ovens,  cooling  air,  and  screen  rooms. 


a.  Determine ;  Score 

(1)  No  special  environment  is  required.  4 

(2)  Forced  air  cooling  or  an  electromagnetically  2 

shielded  enclosure  is  required. 

(3)  Other  special  environment  conditions  are  required.  0 


Stimulus  and  Measurement  Accuracies.  Determine  the  stimulus  and 
measurement  accuracies  required  for  high  confidence  test. 

a.  Determine ;  Score 

(1)  All  tests  can  be  performed  on  ETE/ATE  at  high  4 

confidence  levels;  i.e.,  stimulus  is  adequate 

and  measurement  is  at  least  ten  times  more  accurate 
than  the  tolerance  on  the  UUT. 

(2)  Measurement  is  at  least  three  but  less  than  ten  3 

times  as  accurate  than  the  UUT  tolerance. 

(3)  Measurement  is  between  one  and  three  times  more  1 

accurate  than  the  OUT  tolerance. 


(4)  Stimulus  and/or  measurement  accuracy  is 
inadequate . 


0 


Test  Point  Adequacy.  Determine  if  sufficient  test  points  are  provided 
for  non-ambiguous  fault  isolation  and  for  monitoring  redundant  circuits 
and  BIT  cIj. cults. 


a.  Pet  mine :  Score 

(1)  Redundant  and  BIT  circuits  can  be  fully  tested  4 

and  test  points  at  the  output  of  each  functional 
circuit  permit  direct  non-ambiguous  fault 

isolation. 

(2)  Indirect  (non-signal  tracing)  troubleshooting  3 

and/or  ambiguous  fault  isolation  (within 

permissible  limits  per  AR-10)  is  necessary. 

(3)  Redundant  and  BIT  circuits  not  tested  or  there  C 

is  excessive  ambiguity. 


Test  Point  Characteristics.  Determine  test  point  impedance  and 
voltage  levels. 


a.  Determine ;  Score 

(1)  Voltage  is  less  than  350  VRMS  ahd  impedance  is  4 

compatible  with  the  ETE/ATE  interface.  lru  test 

points  will  drive  up  to  ten  feet  of  properly 
terminated  coaxial  cable. 

(2)  Voltage  dividers  and/or  passive  and  simple  2 

active  impedance  transformation  are  required 

within  the  ID. 

(3)  Waveshaping  and/or  signal  transformation  is  2 

required. 

Test  Point  Isolation.  Determine  if  damage  to  a  UUT  will  result  from 
a  short  circuit  between  any  teat  point  and  ground,  or  if  wideband  noise 
impressed  on  the  test  point  will  degrade  performance. 

a .  Determine:  Score 

(1)  Test  points  are  insensitive  to  external  4 

disturbance  and  no  damage  results  from  a 

short  circuit. 

(2)  Test  points  are  sensitive  to  external  2 

disturbance  but  no  damage  results  from 

short  circuit. 

(3)  A  test  point  short  circuit  will  damiage  the  0 

UUT. 
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Power  and  Load  Requirements.  Determine  the  current  and  voltage  required 
to  power  the  LRU  and  the  loads  required  to  absorb  the  output  power  of 


the  LRU. 

a.  Determine :  Score 

(1)  The  power  and  load  requirements  can  be  met  4 

by  standard  ETE/ATE  resources. 

(2)  The  loads  can  be  accommodated  in  a  simple  or  3 

intermediate  ID. 

(3)  The  quantity  of  loads  is  such  that  the  ID  is  0 

complex  or  ship's  power  or  an  external  power 

source  must  be  used. 


Warm-up.  Determine  if  the  LRU  and/or  any  SRU  requires  warm-up 


on  ETE/ATE  to  ensure  accurate  test. 

a.  Determine:  Score 

(1)  Mo  warm-up  is  required.  4 

(2)  Warm-up  is  less  than  5  minutes.  2 

(3)  Less  than  15  minutes.  1 

(4)  Greater  than  15  minutes.  0 


Evaluation:  Generally,  a  UUT  should  score  31  or  more  total 
points  (of  44  possible)  to  be  considered  acceptable  in  terms 
of  testability.  A  score  of  zero  in  any  of  the  eleven  compatibility 
■elements  indicates  a  need  for  modification  of  the  design. 

Design  Data  Checklist  -  Qualitative  AidB 

General  Interface  Requirements 

a.  Unit-under-test  orientation  and/or  environment  should  be  non-critical. 

b.  All  adjustment  points  should  be  clearly  indicated,  together  with 
adjustment  parameters. 

c.  There  should  be  no  EMX  or  R7X  problems  ir  testing  the  unit. 
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Electrical  Interface  and  Parameters 

a.  Tolerances  or  limits  of  parameters  are  compatible  with  field 
preferably  to  factory  requirements. 

b.  All  special  loading  requirements  are  defined. 

c.  Special  requirements  are  known  for  settling  time  relative  to 
making  a  measurement. 

d.  Primary  power  requirements  are  clearly  indicated,  including 
maximum  allowable  variations  in  voltage  and  frequency. 

Sequence  of  application  or  removal  of  power  is  identified  necessary. 

f.  Each  input  and  output  signal  is  completely  defined. 

g.  Each  test  point  signal  is  completely  defined. 

h.  High  frequency  line- lengths  are  non-critical . 

i.  Trigger  or  synchronizing  inputs  are  provided  from  ETE  or  ATE. 

3.  Completion  Criteria. 

The  task  may  be  considered  complete  when  results  are  incorporated  in  the 
Testability  Analysis  Report,  Task  V5A.  Further  iteration  and  refinement 
will  occur  in  the  Full  Scale  Development  Phase,  Function  FI. 


11-24 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2C 


PHASE: 


FUNCTION: 


TASK  TITLE: 


VALIDATION 

System  Design,  Testability  Incorporation 

Provide  direct  designs  and  test  sequences  using  parts 
selected  for  testability 


TASK  OBJECTIVE:  Assure  that  the  system  design  uses  parts,  straightforward 

equipment  designs,  and  software  that  have  proven  test¬ 
ability  characteristics  in  every  possible  instance. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 

Engineering 

e  Testability  Design 
Guidelines 

e  Design 

Engineering 

e  Logic  diagrams,  parts  Lists 

e  Testability  Analysis 
Results 

e  Application 
Software 

e  Test  Program  Listing 

e  Critique  Listing  for 
Comments,  Traps 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Favorable  parts  selections  and  design 
implementations  serve  to  reduce  life  cycle  costs  of  testing  and  test  support. 
Also,  use  of  consnon  approved  parts  as  feasible  reduces  acquisition  as  well  as 
support  costs. 


TASK  SYNOPSIS; 

1.  Task  Requirements. 

1.1  The  following  should  act  as  a  constraint  against  the  design, 
a.  Parts  Selection. 

In  selecting  between  parts,  each  with  satisfactory  performance 
characteristics,  give  preference  to  integrated  circuit  components 
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and  assembled  modules  which  have  proven  satisfactory  testability 
characteristics,  and  to  those  integrated  circuits  for  which  sufficient 
disclosure  of  internal  IC  structure  and  failure  "ides  have  been  pro¬ 
vided  as  a  basis  for  effective  economical  testing.  (5) 


If  permitted  by  performance  requirements,  provide  structured, 
straightforward  designs  using  standard  components  rather  than  random 
designs  using  nonstandard  components.  In  generating  test  sequences, 
a  regular,  systematic  test  is  preferable  to  a  test  which  employs 
subtle  tricks  for  length  minimization. 


Implementation . 


2.1  Parts  Design  and  Software  Guidelines. 


The  following  guidelines  condensed  from  the  literature-at-large  of 

testability  philosophy,  should  be  used  to  incorporate  parts,  equipment 

designs,  and  software  of  proven  testability  characteristics  into  the 

prime  system  design. 

a.  Minimum  functional,  fault  detection  and  fault  isolation  test  costs 
should  be  considered  a  prime  requirement  during  the  design  stage. 

b.  All  constraints  imposed  upon  the  board  assembly  testing  should  be 
covered  in  the  initial  design  plan  and  not  as  an  afterthought. 

c.  Provide  the  results  of  testability  studies  to  the  designer  in 
addition  to  initial  guidelines. 

d.  The  evaluation  of  hardware  chosen  for  developmental  testing  should 
include  design  and  testability  constraints. 

e.  Where  possible,  use  standard  highly  testable  components  and  consis¬ 
tent  part  orientation. 

f.  Provide  simple  and  short  reset  sequences  which  the  tester  can 
control  (See  Initialization-Validation  phase.  Task  V2E) . 

g.  Provide  tester  access  to  the  main  buses. 

h.  Limit  the  number  of  logic  families. 


i.  Make  testability  requirements  known  to  mechanical,  electrical,  and 
software  design. 
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General  Good  Engineering  Practices « 

a.  Electrical  Design  Ideas 

(1)  Make  system  independent  of  supply-voltage  sequences. 

(2)  Allow  shorting  of  input-output  lines. 

(3)  Allow  for  reasonable  capacitive  loads  for  both  input-output 
lines  and  test  points. 

(4)  Allow  shorting  of  adjacent  connector  pins. 

(5)  Try  to  prevent  "Domino"  failures. 

(6)  Use  only  one  logic  family. 

(7)  Allow  internal  clocks  to  be  easily  disabled. 

(8)  Allow  for  external  initialization. 

b.  Mechanical  Design  Practices 

(1)  Use  zero  insertion  connectors. 

(2)  Use  a  defeatable  keying  system  for  using  extender  cards. 

(3)  Leave  space  between  components. 

(4)  Use  consistent  part  orientation  and  standard  parts. 

(5)  Limit  the  number  of  logic  families  within  each  assembly. 

(6)  Use  functional  packaging. 

c.  Logical  Design  Practices 

(1)  Use  built-in  test  (BIT) . 

(2)  Use  selected  test  and  control  points. 

(3)  Avoid  wired  ANDs  and  ORs  in  system. 

(4)  Use  wired  AND  for  test  purposes. 

(5)  Interrupt  feedback  and  redundant  loops. 

(6)  Break  up  long  counter  chains. 

d.  Managerial  Practices 

(1)  Allow  the  circuit  designer  to  be  responsible  for  tests  and 
test  programs. 

(2)  Use  configuration  control. 

(3)  Establish  achievable  goals. 

(4)  Make  testability  requirements  a  peer  set  with  the  disciplines 
of  reliability,  maintainability,  et  al. 


Criteria. 


When  considering  only  the  inherent  testable  design  aspects  of  testability, 
several  measures  may  be  used  which  give  a  gross  indication  of  potential 
testability  and  point  to  problem  areas.  Most  of  these  simply  make  use 
of  a  checklist  (or  perhaps  a  weighted  checklist)  for  the  presence  or 
absence  of  testable  characteristics.  Por  example:  (7) 

a.  Is  the  design  straightforward  and  regular? 

b.  Is  the  circuit  electrically  and  mechanically  compatible  with  the 
available  ETE/ATE? 

c.  Are  conservative  timing  and  signal  tolerances  used? 
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ISIS! 


d.  Is  the  design  partitioned  by  function? 

s  the  circuit  include  a  master  reset? 
control  and  data  paths  separated? 
critical  nodes  brought  out  to  test  points? 
non-standard  components  used? 
the  test  sequences  regular  and  systematic? 

These  and  similar  elements  should  be  adapted,  tailored  and  applied  as 
criteria  during  working  and  formal  design  reviews. 


Completion  Criteria. 


The  task  i»  completed  for  a  specific  program  phase  when  the  specifications 
and/or  design  documentation  for  that  phase  are  found  to  meet  the  design 
criteria  and  are  accepted  by  the  Government. 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2D 


PHASE » 

FUNCTION: 

TASK  TITLE: 


VALIDATION 

System  Design,  Testability  Incorporation 
Design  units  with  testable  physical  partitioning 


TASK  OBJECTIVE:  Assure  that  the  systematic  partitioning  at  each  successive 

indenture  level  of  the  prime  system  provides  for  ease  of 
functional  testing  and  fault  isolation  as  well  as  for  ease  of  repair  by 
removal  and  replacement. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS : 

T  INTERFACE _ INPUT  TO  T _ OUTPUT  FROM  T 


e  System 
Engineering 

e  Design 

Engineering 


e  System  Weight  &  Size, 
System  Partitioning 

e  Subsystem  Partitioning 


e  Maintainability  e  Maintainability  and 


Engineering 

e  Life  Cycle 
Costing 


Repairability  Requirements 
e  Cost  Analysis,  Tradeoffs 


e  Testability  Require¬ 
ments,  Design  Guidelines 

e  Testability  Require¬ 
ments,  Design  Guidelines 

e  Merge  Maintainability/ 
Testability  Requirements 

e  Critique  of  LCC  Trade¬ 
offs 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Reduced  costs  result  from  proper  parti¬ 

tioning,  with  benefits  beginning  to  manifest  themselves  during  developmental 
test.  Acquisition  cost  reductions  occur  in  the  areas  of  spares  and  test 
program  sets.  Operating  and  maintenance  skill  levels  and  manhour  expendi¬ 
tures  will  also  be  reduced. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  following  should  guide  the  design 

a.  The  physical  partitioning  of  an  equipment  into  modules  will  be 

based,  in  part,  upon  the  enhancement  of  the  fault  isolation  process. 


i 
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b.  The  maximum  number  of  UUT  pins  will  be  consistent  with  the 
interface  capabilities  of  the  proposed  BTE  or  ATE. 

c.  Modules  will  be  designed  to  be  easily  removable  plug-in  units  and, 
whenever  practical  and  economically  justified,  will  be  of  such  cost 
and  reliability  that  they  may  be  considered  for  discard-on-failure. 

d.  Any  individual  module  should  be  limited  to  one  circuit  technology, 
containing  only  analog  or  only  digital  circuitry,  whenever  practical. 

e.  Where  practical,  circuits  belonging  to  an  ambiguity  group  will  be 
placed  in  the  same  package  (component,  module) . 


2.  Implementation . 


2.1  Physical  partitioning  is  that  step  in  the  design  process  which  provides 
a  system  that  is  physically  and  functionally  packaged  to  facilitate 
testability  and  maintainability.  This  step  normally  occurs  after 
completion  of  system  logic  design.  It  is  accoeqalished  by  breaking  the 
complete  system  into  smaller  subsystems  which  are  then  packaged 
individually. 

Constraints  include  a  limit  on  the  size  and  number  of  conponents  in  a 
subsystem  as  well  as  on  the  number  of  interconnections  between  subsystems. 

2.2  Appendix  A*75  is  a  (condensed)  heuristic  method  for  enhancing  testability 
during  the  partitioning  of  a  large  digital  system.  The  test  flow  model 
can  be  used  for  test  point  insertion  and  can  easily  be  incorporated  into 
other  partitioning  procedures  which  optimize  primary  constraints  such  as 
the  number  of  elements  or  the  number  of  I/O  pins. 

2.3  One  objective  of  physical  partitioning  is  optimized  system/black  box 
repairability. Three  important  avionics  equipment  repair  problem 
areas  to  which  this  concept  is  applicable  ares  excessive  subsystem 
packaging  complexity,  excessive  black  box  weight,  and  black  box 
troubleshooting/repair . 

Fewer  black  boxes  for  minimum  system  packaging  complexity  benefits 
primarily  the  organizational  maintenance  level  with  major  reductions 
in  organizational  level  support  requirements  in  the  areas  of  labor, 
training  (especially  OJT) ,  technical  data,  and  supply. 

To  avoid  excessively  heavy  black  boxes,  partitioning  units  into  two  or 
more  units  can  result  in  a  weight  such  that  handling  equipment  would 
not  be  required  and  manhandling  (the  preferred  alternative)  would  be 
facilitated.  Partitioning  subsystems  into  packages  which  are  easily 
handled  by  two-man  teams  would  benefit  all  maintenance  levels.  Special 
handling  equipment,  as  presently  authorized  for  organizational  level 
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handling  of  heavy  units,  would  not  be  required  in  the  future,  thereby 
significantly  reducing  support  equipment  requirements. 


Optimized  partitioning,  in  conjunction  with  new  interface  concepts  and 
integrated  electronics,  can  be  applied  to  the  design  process  to  simplify 
troubleshooting  and  repair  in  the  intermediate  shop.  Complex  jnits  such 
as  presently  employed  in  the  weapons  control  and  weapons  delivery  equip¬ 
ment,  are  difficult  to  troubleshoot  to  the  module  level.  This  problem 
is  a  result  of  various  factors  including  number  of  modules  per  black  box, 
chassis  complexity,  and  test  equipment  limitations.  Partitioning  into 
fewer  modules,  and  in  some  cases  eliminating  the  chassis  and  using 
cable  assemblies  for  module  interconnections  (example  is  the  AN/ARC- 164 
radio  design) ,  can  greatly  simplify  unit  troubleshooting.  Further  par¬ 
titioning  of  modules  into  smaller  plug-in  throwaway  sub-modules  (as 
used  in  the  Mark  V  receiver/transmitter)  is  an  additional  refinement  of 
the  partitioning  concept  for  further  simplification  of  the  repair  process. 
Such  partitioning  concepts  would  simplify  troubleshooting/repair  and 
would  have  significant  impact  on  the  intermediate  and  depot  levels  of 
maintenance,  primarily  in  the  area  of  labor. 


2.4  Quality  Criteria. 


The  following  relevant  parameters  may  be  adapted  quantitatively  or  quali¬ 
tatively  to  formulate  criteria  for  measuring  the  effectiveness  of  the 
partitioning  task  as  tailored  to  specific  programs  and  systems. 


PARTITIONING  RELEVANT  PARAMETERS 

e  Element* 

Quantities 


e  Element* 

Weights 

e  Element* 

Volumes 

e  Element* 

Complexities 

e  Number  of  Interconnections 
(I/O  Pins) 

3.  Completion  Criteria. 

The  task  must  be  considered  coop) 
of  Task  V6  have  been  approved  by 


POSITIVE  EFFECT _ 

e  Simplified  Troubleshooting/Repair 
Procedures 

e  Minimized  Organizational  Level 
Test  Support  Requirements 

e  Obviated  Special  Handling 
Equipment 

e  Easier  Handling 


e  Simplified  Troubleshooting/Repair 
Procedures 

e  Less  complex  Interface,  Improved 
Testability 


when  the  development  specifications 
Government . 


*  "Element"  may  be  construed  as  a  OUT  at  any  level  of  hardware  hierarchy. 
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APPENDIX  V2D1*7* 

MODELING  THE  SYSTEM 

Assume  that  the  system  is  described  as  a  set  of  interconnected  elements,  each 
having  its  own  input  and  output  pins.  In  addition,  the  system  itself  will 
have  a  set  of  primary  inputs  and  outputs.  All  interconnections  within  the 
system  will  be  one  of  three  types:  (1)  a  primary  input  to  an  element  input, 

(2)  an  element  output  to  a  primary  output,  or  (3)  an  element  output  to  an 
element  input.  (See  Figure  1.)  Assume  that  all  testing  of  the  system  must 
be  accomplished  through  the  primary  inputs  and  outputs. 

The  functional  nature  of  the  elements  will  not  be  considered;  they  may  range 
from  simple  gates  up  to  entire  cabinets.  Each  element  will  be  assigned  a 
measure,  t*,  which  reflects  its  testability  relative  to  the  other  elements 
in  the  system.  The  nature  of  this  "measure  of  testability"  is  left  to  the 
user.  It  could  be  as  sophisticated  as  the  number  of  tests  required  to  detect 
some  fixed  percentage  of  all  stuck-at  faults  in  that  element  or  as  simple  as 
a  mere  component  count,  as  long  as  it  accurately  reflects  the  relative  test 
requirements  of  the  elements.  Figure  1  shows  a  set  of  such  measures  assigned 
to  the  elements  of  the  system  with  a  "test  load"  of  1300. 


Figure  1.  Model  of  a  System 
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Secondly,  consider  the  accessibility  of  these  elements  ns  viewed  from  the 
primary  inputs  and  outputs.  An  element  with  a  relatively  small  testability 
measure  may  be  quite  difficult  to  test  simply  because  it  is  buried  deep 
within  the  system. 

Picture  the  testing  of  an  element  (with  measure  t*)  as  a  flow  process:  a  set 
of  t*  input  tests  or  excitations  are  introduced  onto  the  primary  inputs  of 
the  system.  These  excitations  propagate  through  the  system  to  the  inputs  of 
the  element  involved.  They  pass  through  the  element  and  emerge  on  the  outputs 
as  a  set  of  t*  detections.  These  detections  then  continue  on  to  the  primary 
outputs  of  the  system  where  they  can  be  examined  for  indications  of  possible 
malfunctions . 

The  nature  of  the  flow  will  change  (from  excitations  to  detections)  when  it 
passes  through  the  element. 

We  make  one  further  assumption  that  no  feedback  paths  be  present.  The  parti¬ 
tioning  constraints  prohibit  the  cutting  of  some  or  all  of  such  feedback  paths. 
Now  all  elements  appearing  in  these  paths  can  be  merged  into  a  single  element 
whose  testability  measure  is  then  adjusted  accordingly.  (Ptraightforward 
techniques  for  accomplishing  this  are  described  by  Ramamoorthy  &  Chang*.)  If 
feedback  paths  still  exist,  various  interconnections  may  be  tenporarily 
deleted.** 

First  distribute  the  t*  detections  coming  out  of  the  element  forward  to  the 
primary  outputs  using  the  following  "divide  equally"  assumptions: 

(1)  The  t*  detections  are  assumed  to  divide  equally  among  the  output  pins  of 
the  given  element. 

(2)  If  an  output  pin  drives  a  primary  output,  all  of  its  flow  will  be  assumed 
to  go  to  that  output.  Otherwise,  it  will  be  assumed  to  divide  equally 
among  the  input  pins  which  it  drives. 

(3)  The  total  flow  into  any  other  element  will  be  assumed  to  divide  .equally 

among  that  element's  output  pins.  , 

f 

Figure  2  shows  the  application  of  these  rules  to  the  200  unit  flow  out  of 
element  A.  The  flow  divides  into  100  on  each  of  the  two  output  pins.  These 
flows  then  continue  to  the  inputs  of  elements  B  and  C.  In  element  B,  the  100 
unit  flow  again  divides  equally  with  50  units  going  on  to  a  primary  output 
and  50  units  to  element  C.  Finally,  the  total  flow  of  150  into  C  divides 
equally  between  its  two  output  pins  and  continues  to  the  respective  primary 
outputs . 


*  C.V.  Ramamoorthy  &  L.C.  Chug,  "System  Segmentation  for  the  Parallel 

Diagnosis  of  Coaputers",  IEEE  Trans,  on  Computers,  Vol.  CT-20,  pp  261-270, 
March  1971. 

**  U.  R.  Kodres,  "Partitioning  and  Card  Selection”  in  Design  Automation  of 
Digital  Systems  (M.  Breuer,  ed.)  Englewood  Cliffs,  N.J.:  Prentice-Hall, 
1972. 


V2D1-2 


The  set  of  "divide  equally"  assumptions  nay  be  used  to  propagate  the  200 
excitations  of  element  A  backwards  to  the  primary  inputs.  Figure  3  shows 
the  result  of  this  step.  (Detections  have  been  underlined  to  distinguish 
them  from  excitations.) 


Figure  2.  Distributing  Detections 


Now  repeat  this  flow  generation  process  fcr  each  of  the  elements  within  the 
system.  Then  add  up  the  resulting  flows  to  obtain  the  completed  test  flow 
model  shown  in  Figure  4.  The  flow  on  each  lead  is  designated  by  two  numbers 
denoting  the  number  of  detections  (underlined)  and  the  number  of  excitations 
which  that  lead  will  carry. 
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THE  TEST  FLOW  MODEL 


Note  that  for  any  element  with  meaaure  t*s 


d  »  d.  +  t* 
out  in 


(1) 


and, 


e  .  *  e.  -  t* 

out  in 


(2) 


where  d  and  e  denote  detections  and  excitations. 


EXCITATIONS 


Figure  3.  Distributing  Excitations 


1 
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Figure  4.  The  Test  Flow  Model 


For  example,  the  flow  into  element  B  (Figure  4)  is  520,  220  detections  and 
300  excitations.  The  total  out  is  also  520  composed  of  460  detections  and 
60  excitations  -  the  result  of  240  excitations  changed  to  detections. 

If  we  partition  the  elements  of  the  system  into  two  disjoint  sets,  then  the 
set  of  leads  interconnecting  these  two  sets  comprise  a  cut.  See  (*)  for  a 
precise  definition.  The  flow  across  a  cut  is  equal  to  the  total  flow,  T, 
through  the  entire  system.  For  any  cut,  C,  if  we  add  up  the  detections  and 
excitations  on  its  individual  leads,  then 

Idi  +  I*i  "  T  (3) 

c  c 

Consider  the  6  lead  cut  in  Figure  5.  The  sum  of  the  detections  is 
120+160+160060+380  •  880.  The  sum  of  the  excitations  is  60601001001000 
-  420. 

T  -  880  +  420  ■  1300 


(*)  c. Barge,  The  Theory  of  Graphs,  New  York:  John  Wiley  C  Sons,  1962 
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The  detection  flow  of  880  equals  the  sum  of  the  measures  of  the  four  elements 
above  the  cut  and  the  excitation  flow  of  420  corresponds  to  the  sum  of  the 
measures  of  the  two  elements  below  the  cut. 

The  individual  detection  and  excitation  flows  tell,  respectively,  exactly  how 
this  load  is  divided  above  and  below  the  cut.  This  properly  can  be  especially 
useful  during  the  partitioning  process. 


Figure  5.  A  Cut 


We  wish  to  partition  the  system  in  figure  4  into  two  subsystems  so  that  the 
total  test  load  is  divided  as  nearly  equal  as  possible  between  the  two  sub¬ 
systems  . 

Assign  each  lead,  i,  a  weight,  w^,  which  is  equal  to  the  absolute  value  of  the 
difference  between  its  detections  and  its  excitations,  i.e., 

wA  -  (dA  -  e^  (4) 
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Figure  6  shows  the  lead  weights  which  would  result  in  this  example.  Note: 
leads  near  the  center  of  the  system  tend  to  have  snail  weights,  those  near  the 
extremes  have  large  weights.  The  value,  c  of  any  cut  -  i . e . ,  the  sum  of  the 
weights  of  the  leads  in  the  cut  -  gives  any  upper  bound  on  the  difference  of 
Ti  and  T2  (the  test  loads  of  the  resulting  subsystems) : 

|Ti  -  T2|S  c 


Figure  6.  Weighted  Leads 

If  we  find  the  minimum  cut  of  the  system  (with  value  c*) ,  then  the ' resulting 
test  loads  will  differ  by  at  most  c* . 

A  basic  algorithm  from  network  flow  theory  (The  Min  Cut/Max  Flow  Algorithm) 
finds  precisely  this  cut. 
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Figure  7  shove  the  minimum  cut  which  results  when  this  algorithm  is  applied 
to  the  example.  The  test  load  divides  as: 

*  680  and  T2  “  620. 


Figure  7.  The  Minimum  Cut 


This  8c ne  technique  (weighting  the  leads  and  then  using  the  Min  Cut/Max  Flow 
algorithm)  can  be  generalised  to  partitioning  the  system  into  p  parts  by 
simply  adjusting  the  weighting  procedure.  Define  each  lead  weight  as: 

wi  "  |'p  ~  1)di  "  ®i  |  *  (6) 

The  test  load  above  the  resulting  minimum  cut  will  be  roughly  T/p,  where  T  is 
the  total  test  load.  Once  the  subsystem  generated  by  this  cut  is  removed,  the 
test  flow  for  the  remainder  of  the  system  may  be  recalculated  and  the  process 
repeated  for  p-1,  etc.  A  computer  program  implementing  this  procedure  has 
proved  to  be  quite  fast  and  effective.  Its  arc.  is  illustrated  below. 
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Figure  6  shorn  the  lead  weights  which  would  result  in  this  example.  Note: 
leads  near  the  center  of  the  system  tend  to  have  snail  weights,  those  near  the 
extremes  have  large  weights.  The  value,  c  of  any  cut  -  i.e.,  the  sum  of  the 
weights  of  the  leads  in  the  cut  -  gives  any  upper  bound  on  the  difference  of 
T]_  and  T2  (the  test  loads  of  the  resulting  subsystems) : 

|ti  -  t2|s  c 


Figure  6.  Weighted  Leads 


If  we  find  the  minimum  cut  of  the  system  (with  value  c*) ,  then  the  resulting 
test  loads  will  differ  by  at  most  c*. 

A  basic  algorithm  from  network  flow  theory  (The  Kin  Cut/Max  Flow  Algorithm) 
finds  precisely  this  cut. 
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Figure  7  show*  the  minimum  cut  which  results  when  this  algorithm  is  applied 
to  the  example.  The  test  load  divides  ass 

-  680  and  T2  ”  620. 


Figure  7.  The  Minimum  Cut 


This  same  technique  (weighting  the  leads  and  than  using  the  Min  Cut/Max  Flow 
algorithm)  can  be  generalized  to  partitioning  the  system  into  p  parts  by 
simply  adjusting  the  weighting  procedure.  Define  each  lead  weight  as: 

wi  “  l(p  "  1)di  "  ®i  |  '  (6) 


The  test  load  above  the  resulting  minimum  cut  will  be  roughly  T/p,  where  T  is 
the  total  test  load.  Once  the  subsystem  generated  by  this  cut  is  removed,  the 
test  flow  for  the  remainder  of  the  system  may  be  recalculated  and  the  process 
repeated  for  p-1,  etc.  A  computer  program  implementing  this  procedure  has 
proved  to  be  quite  fast  and  effective,  its  use  is  illustrated  below. 
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Partitioning  problams  usually  involve  a  number  of  considerations  which  must  be 
examined.  Two  of  the  mosc  conmon  are  the  total  sice  of  the  elements  in  each 
subsystem  and  the  number  of  interconnections  between  subsystems.  To  incor¬ 
porate  size  considerations  into  the  Min  Cut  procedure  initially  distribute 
the  numerical  size*  •*,  of  each  element  through  the  system  just  as  we  did  for 
the  t*s.  Each  lead  i  will  then  end  up  not  only  with  a  d^  and  e^,  but  also 
with  an  si  and  an  s'i.  Across  any  cut Vsi  will  equal  the  total  size  of  the 
elements  above  the  cut  and£s'^  the  total  below  the  cut.  If  a  partition 
which  has  been  chosen  to  optimize  testability  la  found  to  violate  a  size  con¬ 
straint,  then  the  lead  weights  may  be  changed  to  a  weighted  sum  of  the  s' s 
as  well  as  ths  d's  and  e's.  The  exact  nature  of  this  sum  will  depend  on  botn 
the  desired  size  of  the  partition  and  on  its  criticality  relative  to  test¬ 
ability  considerations. 

When  the  number  of  leads  in  an  indicated  cut  is  found  to  be  excessive,  a 
properly  chosen  weight,  1,  can  be  added  to  the  other  lead  weights  so  that  the 
resulting  minimum  cut  will  tend  to  minimize  the  number  of  leads  as  well. 

TEST  POINT  INSPECTION 

> 

On  every  primary  input  pin  we  have  a  numerical  indication  of  its  share  of 
the  input  excitation  load,  and  the  detection  load  on  the  primary  output 
pins. 

Consider  the  test  flow  model  of  Figure  4,  and  assume  that  this  represents 
a  subsystem  resulting  from  the  partitioning  process.  If  we  are  to  insert 
a  single  test  point  where  should  this  be  dene  to  best  enhance  the  subsystem’s 
testability? 

Choose  the  internal  lead  that  carries  the  largest  number  of  detections.  In 
this  case,  the  internal  lead  from  B  to  C  (with  230)  would  be  chosen  and  the 
test  point  inserted  accordingly.  Modify  the  model  to  reflect  an  insertion 
at  this  location.  By  following  the  same  rules  by  which  it  was  originally 
distributed  and  decrementing  the  flows  accordingly,  the  resulting  test  flow 
is  that  shown  in  Pigure  8. 


Another  possibility  is  to  consider  "cutting"  an  internal  lead  -  i.e,  to 
replace  the  lead  by  two  test  points,  one  a  primary  input  and  the  other  a 
primary  output.  During  normal  operation  these  points  would  be  -interconnected, 
but  removed  during  the  testing  phase.  A  good  candidate  for  such  a  cutting 
process  is  that  internal  lead  with  the  largest  total  test  flow.  In  Figure  4, 
we  select  the  internal  lead  into  element  A  having  a  total  flow  of  300. 

Figure  9  shows  the  resulting  test  flow. 
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Figure  8.  Adding  a  Test  Point 


Figure  9.  Cutting  a  Lead 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2E 


PHASE:  Validation 

FUNCTION :  System  Design,  Testability  Incorporation 


TASK  TITLE:  Incorporate  initialization  into  design 

TASK  OBJECTIVE:  Assure  that  the  design  provides  circuitry,  firmware  and 

software  or  application  programs  to  establish  a  well- 
defined  unique  initial  or  starting  state  at  the 
beginning  of,  or  at  prescribed  points  in,  a  functional 
test  or  a  fault  isolation  process. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER- RELATIONSHIPS : 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 

a  System  level  BIT 

a  Initialization 

Engineering 

data 

Requirements 

•  Design 

•  Number  &  placement  of 

a  Initialization 

Engineering 

UUT  test  points  and 

Requirements 

test  control  inputs 

e  Application 

e  BIT  programs 

Software 

e  Life  Cycle 

a  Cost  guidance 

a  ATE  software 

Cost 

tradeoffs 

e  System 

a  Evaluation  of 

a  T  demonstration 

Test 

demonstrability  of 

candidate  elements 

initialization 

COST  TRADEOFF  INTER-RELATION^  HIPS ;  Well-designed  initialization  capability 

reduces  ETE/ATE  and  BIT  software  costs 
and  reduces  the  costs  associated  with  field  maintenance  testing. 

TASK  SYNOPSIS: 

1.  Task  Requirements . 

Initialization  requirements  pertain  primarily  to  digital  applications. 


^TSPfp'epip’*' 
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although  analogous  requirements  may  well  exist  to  establish  a  basic 
state  for  analog  testing. 

1.1  Two  features  which  characterize  initialization  in  the  design  are: 

a.  The  system/equipment  design  should  be  such  that  it  has  a  well- 
defined  initial  state  to  commence  the  fault  isolation  process. 
Non-achievement  of  the  correct  initial  state  should  be  annunciated 
to  the  operator  along  with  sufficient  signature  data  for  fault 
isolation. 

b.  The  system/equipment  should  be  designed  to  initialize  to  a  unique 
state  such  that  it  will  respond  in  a  consistent  manner  for  multiple 
testing  of  a  given  failure.  (5) 

2.  Implementation . 

2.1  The  following  are  suggested  techniques  for  achieving  initialization 
of  the  logic  elements  and  are  not  all  inclusive.  (10) (11) 

a.  Use  wired  AND  (OR)  for  external  control  connections  (Figure  1)  as 
a  means  of  initializing  the  logic  elements.  This  wired-or 
technique  is  safe  for  DTL  circuits  and  for  standard  and  low-power 
TTL  circuits.  But  high-pcwer  and  Schottky  TTL  circuits  cannot 
be  driven  low  safely  for  more  than  one  second.  Where  an  output 
of  an  IC  feeds  back  into  the  IC  (as  in  a  flip-flop,  shift  register 
or  counter) ,  the  output  may  be  driven  low  using  the  wired-or 
technique.  For  example,  the  "slave"  stage  of  a  master- slave 
flip-flop  can  be  initialized  by  driving  the  Q  or  Q  output  low, 
but  the  "master"  stage  remains  unchanged.  See  Figure  2. 


WO  MW 
OR 

TUT  POINT 


o 

0» 


Figure  1.  Wired  OR  for  Tett  Control  Input 
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Figure  4.  Power-Up  Reset 


d.  The  same  pull-up  resistor  can  be  used  for  several  sets  or  resets 
on  different  memory  elements .  (Figure  5) . 


Figure  5.  Pull-Up  Resistor  Multiple  Reset 


e.  If  a  set-reset  line  is  tied  directly  to  V  or  ground,  it  cannot 
be  driven  by  the  teste’-.  Instead,  a  source  of  a  logic  high  should 
come  from  a  pull-up  resistor  and  a  source  of  a  logic  low  should 
come  from  an  inverter  whose  input  is  pulled  high. 

2.2  A  verification-of  design-review  is  necessary  since  initialization 

cannot  be  fully  validated  until  the  performance  of  a  maintainability 
or  testability  demonstration.  Internal  low-level  design  reviews 
should  therefore  be  conducted,  either  by  supervision  or  by  one 
designer  reviewing  another's  work.  The  baseline  verification  guide  is 
to  verify  that  all  memories,  flip-flops  and  registers  can  be 
initialized  to  a  known  state. 


Coordination  should  also  be  effected  with  the  planning  for  the  test¬ 
ability  or  maintainability  demonstration  for  evaluation  of  the 
demonstrability  of  the  initialization  characteristics  of  the  system. 


2.3  A  procedure  should  be  tailored  to  each  design  program  to  obtain 
certification  as  a  result  of  internal  design  review  processes  and 
incorporation  of  initialization  measurement  into  the  I  or  M 
demonstration  planning. 

3.  Completion  Criteria. 

The  task  is  considered  complete  when  the  development  specifications 
(Task  V6)  are  approved  by  the  government. 


TESTABILITY  TASK  COMPENDIUM 
Task  Rsfsrsncs  Number  V2F 


PHASE: 


VALIDATION 


FUNCTION: 

TASK  TITLE: 

TASK  OBJECTIVE: 


System  Design,  Testability  Incorporation 

Incorporate  controllability  into  design 

Assure  the  design  provides  adequate  test  control  over 
system/UTJT  operation. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  Design 

Engineering 

e  Logic  Diagrams,  Schematics 

e  Controllability  Require¬ 
ments  &  Guidelines 

e  Application 
Software 

e  Test  Programs,  Logic  Flow 
Diagrams 

e  Controllability  Require¬ 
ments  &  Guidelines 

e  Life  Cycle 

Cost 

e  LCC  Information 

e  Test  Cost  Parametric 

Data 

COST  TRADEOFF  INTER-RELATIONS HIPS :  The  degree  of  controllability  built  into 

the  hardware  and  software  design  results  in  proportionally  less  complex  test 
equipment  and  test  programs  with  reduced  acquisition  costs  therefor.  Also, 
downstream  operations  and  maintenance  costs  associated  with  reduced  test  and 
repair  times  and  corresponding  maintenance  labor  and  spares  reductions  pro¬ 
vide  further  LCC  savings. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  Controllability  is  defined  as  "an  attribute  of  equipment  design  which 
defines  or  describes  the  degree  of  test  control  which  may  be  exercised 
at  internal  nodes  of  interest. "CD 

This  task  guides  the  designer  in  developing  circuitry  such  that  BIT  and/ 
or  BITE  and/or  ETE  simplifies  testing  via  controls  over  internal  module 
and  component  operation.  The  requirement  is  for  valid  detection  and 
isolation  of  faults  and  failures. 

2.  Implementation . 

Controllability  of  test  is  accomplished  essentially  at  internal  nodes; 
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therefore,  the  implementation  techniques  provide  for  access  where 
necessary  for  additional  data  paths  and  circuitry.  The  literature  is 
fairly  extensive  in  coverage  of  techniques  applicable  to  digital  cir¬ 
cuitry,  while  analog  test  is  covered  by  inference  or  by  references  to 
textbooks . 

2.1  The  following  list  of  controllability  design  guidelines  apply  to  digital 
equipment. (12) 


a.  A  digital  circuit  design  should  avoid  large  memory  devices  such  as 
1024  bit  or  larger  RAMs  plus  significant  amounts  of  standard  logic, 
since  such  devices  require  special  test  techniques  and  equipment. 

b.  Dynamic  devices  which  require  periodic  refresh,  e.g. ,  dynamic 
memories  and  some  microprocessors,  require  special  tester  circuitry 
and  should  be  avoided.  Provision  of  a  separate  clock  for  these 
devices  improves  testability. 

c.  Complex  LSI  circuits  (microprocessors,  I/O  controllers,  etc.)  should 
be  installed  in  sockets  because  existing  logic  testers  are  not  satis¬ 
factory  for  testing  boards  with  such  devices.  If  sockets  are 
prohibited,  provide  self-test  capability  on  the  card  or  provide 
electrical  access  to  the  input  and  output  lines  of  the  LSI  devices. 

If  sockets,  self-test,  and  electrical  access  are  all  impractical, 
recognize  that  card  test  will  be  expensive. 

d.  All  input-output  signals  should  be  TTL  compatible.  Interface  cards 
which  use  non-TTL  signal  levels  are  normally  considered  "analog" 
cards.  Testers  include  pull-up  resistors  so  open  collector  outputs 
may  be  used.  The  control  line  for  tri -state  busses  should  be 
externally  accessible.  Input  signals  including  clock  and  master 
clear  require  20  milliantperes  or  less. 

e.  Oscillators  and  monostables  should  be  implemented  so  that,  a)  they 
can  be  electrically  disconnected  using  external  logic  signals,  and 
b)  paths  are  provided  for  external  injection  of  the  functions  they 
would  normally  provide.  Routing  a  signal  off  the  board  and  back  in 
via  I/O  pins  is  smother  way  to  satisfy  these  requirements. 

f.  Large  feedback  loops,  long  logic  paths,  and  long  counters  (more  than 
8  bits)  should  be  broken,  preferably  at  an  I/O  connector  or  by  an 
externally  controlled  logic  signal.  Test  points  can  provide  control 
inputs  or  signal  outputs  to  break  loops,  counters,  and  paths.  IC 
sockets  without  inserted  components  can  provide  interior  test  points; 
by  the  removal  of  jumpers  they  permit  paths  to  be  broken. 

g.  Do  not  use  logic  which  depends  on  specific  clock  rates,  or  controlled 
rise  and  fall  times,  or  on  specific  gate  propagation  delays. 

PROMs  and  ROMs  should  have  electrical  access  provided  to  their  con¬ 
trol  and  output  lines  or,  if  this  is  impossible,  should  be 
removable  from  sockets. 
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2.2  Appendices  1  and  2  contain  design  techniques  applicable  to  discrete 

digital  and  digital  ZC  circuits.  I1*)  Appendix  2  guidelines  are 

especially  applicable  to  compatibility  with  automatic  digital  test  tech¬ 
niques. 

2.3  Testing  a  nonremovable,  free-running  oscillator  is  often  a  problem 
(Figure  1) . 114)  one  solution,  if  the  oscillator  can  withstand  having  its 
output  shorted  to  ground  for  several  seconds,  is  shown  in  Figure  2. 

Table  1  describes  the  circuit  action.  Successful  testing  depends  upon 
placing  the  74S65  as  close  to  the  flip-flop  as  possible.  One  method  is 

to  mount  the  74S65  on  a  small  PC  card  soldered  to  a  dip-clip  that  attaches 
to  the  flip-flop. 


Figure  1.  Problem:  Nonremovable  oscillator  drives  board 
clock  circuitry. 
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Table  1.  Tbis  function  table  describes  the 
action  of  the  control  circuitry. 


Figure  2.  Solutiort:  The  74S65  controls 
the  oscillator  using  feedback  from  the 
flip-flop  and  control  lines  from  the  tester. 


If 
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2.4  Design  for  "Easy"  Testability5^  suggests  some  general  rules  when 
designing  a  board  for  testability  and  gives  specific  advice  for  micro¬ 
processor  based  designers  (ROM  &  RAM) . 

2.5  "Designing  MPU  Boards  for  Testability"  (16)  reviews  basic  rules  that 
designers  should  follow  and  discusses  special  problems  presented  by 
microprocessors ,  current  strategy  for  testing  microprocessor-based 
boards  and  self-test  techniques.  The  board  used  as  an  example  for 
discussion  implements  the  IEEE  STD  488-1975  digital  interface  and  is 
based  upon  the 8080  microprocessor  (contains  RAM,  RDM  and  MSI).  The 
subjects  discussed  are: 

a.  basic  design  rules 

b.  clock-circuit  control 

c.  access  points 

d.  unused  pins 

e.  testing  problems  presented  by  MPUs 

f.  microprocessor  test  strategy 

g.  designing  a  testable  MPU-based  board 

2.6  "Self  Testing  VLSI  Circuit  Packs"  describes  methods  of  implementing 
self-test  in  VLSI  digital  circuit  packs,  assesses  cost  factors  and 
presents  a  design  strategy  that  minimizes  overhead  (minimum  amount  of 
circuitry  devoted  solely  to  test) ,  changes  (minimum  change  in  established 
logic  design  procedures)  and  memory  (minimum  storage  required  for 
testing) . 

2.7  Quality  Criteria. 

There  are  no  firm  and  simple  criteria  for  the  degree  of  controllability 
to  be  built  into  a  unit.  Since  controllability  is  built  into  the  design, 
the  contractor  and  government  program  managers  should  base  reviews  of 
completeness  on  the  guidelines  stated  in  this  synopsis,  applied  at  appro¬ 
priate  internal  anc  formal  design  reviews. 


3.  Commie -ion  Criteria. 

The  task  is  considered  complete  upon  government  acceptance  of  the 
specifications  prepared  in  Task  V6. 
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APPENDIX  V2F1 


(11) 


1.  Simplified  testing  of  complex  sequential  circuits. 

In  order  to  control  a  sequential  circuit  and  test  *  he  operation  of  part 
of  the  circuit,  additional  elements  can  be  used  to  force  a  certain  state 
and  ease  testing.  In  the  case  of  a  latch  extra  inputs  control  the  latch 
externally  (Figure  1) .  Where  a  point  is  enabled  by  many  conditions  an 
extra  input  can  enable  this  point  externally  (Figure  2) . 


2.  Interrupted  feedback  loops. 

a.  Provide  a  means  of  inhibiting  the  clock  of  each  memori  element  to 
break  the  feedback  and  to  provide  known  reference  points.  Also  add 
test  points  to  each  memory  element  in  the  feedback  loop  to  improve 
visibility  in  the  loop. 

b.  Insert  an  extra  gate  in  the  feedback  path  to  interrupt  feedback. 

The  gate  is  controlled  by  a  signal  from  the  tester  (Figure  3). 

c.  Physically  break  the  feedback  loop  and  briny  both  sides  out  to 
external  pins,  which  are  shorted  by  a  jumper  for  normal  operation. 
Removing  the  jumper  interrupts  the  feedback  .nd  provides  both  a 
driving  point  and  a  sensing  point  (Figure  4) . 
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d.  Drive  any  desired  test  point  low  using  the  wired  OR  technique.  When 
not  being  driven,  the  test  point  may  be  used  for  sensing. 


Figure  3.  Gate  Interrupt 


Figure  5.  Test  and  Control  Point 


Test  and  control  point  locations. 

a.  At  junctions  of  large  fan-ins  or  fan-outs  (Figure  5) 

b.  At  eatputs  of  memory  elements 

c.  In  buried  logic 

d.  At  internal  branches  of  statically  redundant  logic 
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Reduced  number  of  test  patterns. 

a.  Add  control  signals  to  the  direct  load  lines  of  long  counter  chains 
where  the  counters  are  directly  loadable. 

b.  Break  the  cascading  between  counter  stages  to  allow  each  stage  to  be 
clocked  independently  with  a  minimum  of  clock  pulses.  The  following 
techniques  may  be  used: 

o  Attach  a  driver  to  pull  down  the  cascade  line,  but  use 
caution  to  avoid  impairing  the  counter  internally. 

o  Physically  interrupt  the  cascade  lines  by  inserting  pairs 
of  test  points  which  are  shorted  by  a  jumper  during  normal 
operation . 

o  Add  extra  logic  to  allow  counter  stages  to  be  clocked 
independent  of  the  carry-out  from  earlier  stages.  For 
exan$>le,  if  the  carry-out  below  is  active  in  the  low  state, 
the  addition  of  am  extra  gate  will  allow  the  following 
stage  to  be  clocked  independently  (Figure  6) . 


Figure  6.  Counter  Stage  Control 
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APPENDIX  V2F2  ^13) 

Appendix  2  consists  of  11  self-explanatory  illustrations. 


Figure  1.  Insure  all  memory  elements  (flip-flops)  can  be 
preset  into  a  known  state  at  the  beginning  of  the  text. 


Figure  2.  Allow  all  free  running  clocks  to  be  disabled 
from  connector  and  the  tester  clock  to  be  inserted  in 
place  of  the  free  running  clock. 


Figure  3.  Avoid  using  circuits 
that  produce  narrow  pulse 
widths  (less  than  500  ns). 


Figure  4.  Avoid  use  of 
"wired  AND”  logic. 
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use  D  type  instead. 


Figure  9.  Provide  the  ability  to  break  logical 
feedback  loops  at  the  connector  of  the  board. 


Figure  10.  On  CPU  boards,  bring  the  address 
and  data  bus  lines  to  an  edge  connector  or  an 
internal  test  connector.  Also,  make  any  hand¬ 
shake  signals  available  to  the  test  programmer. 
Ensure  the  ability  to  operate  the  CPU  board 
from  an  external,  tester  controlled  dock 
source. 
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PHASE: 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2G 

VALIDATION 


FUNCTION: 


System  Design,  Testability  Incorporation 


TASK  TITLE:  Incorporate  observability  into  design 

TASK  OBJECTIVE:  Assure  that  the  design  provides  sufficient  signature  data 

through  test  points,  data  paths,  and  fault  isolation 
circuitry  for  the  test  system  (BIT  or  ETE)  to  discern  and  isolate  faults  (or 
validly  assure  fault-free  condition)  within  the  module (s) . 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  Design 

e  Number  &  Placement  of  UUT 

e  Observability 

Engineering 

Test  Points,  HW  BIT 

Requirements 

Features 

e  Application 

e  Software  BIT  Features 

Software 

e  Life  Cycle 

e  Cost  Guidance 

e  ETE  Tradeoffs  and 

Cost 

Parametric  Cost  Data 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Observability  reduces  the  complexity  of 

the  test  interface  and  therefore  reduces  the  costs  of  TPS  acquisition,  and 
of  skill  levels  and  manhour  expenditures  in  operations  and  maintenance. 

TASK  SYNOPSIS: 

(51 

1.  Task  Requirements. '  ' 

1.1  The  ability  for  external  systems  to  perceive  the  function  or  non-function 
of  module  internal  circuits  is  the  module's  observability.  The  observ¬ 
ability  requirement  is  to  incorporate  test  points,  data  paths,  and 
circuitry  to  provide  the  test  system,  whether  BIT  or  ETE,  sufficient 
signature  data  for  failure  detection  and  isolation  within  the  module. 

The  selection  of  physical  (real)  test  points  should  be  sufficient  to 
accurately  infer  the  value  of  internal  nodes  (virtual  test  points)  of 
interest.  There  should  be  no  requirement  to  probe  internal  points  for 
organisational-level  fault  isolation. 


i 
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2.  Implementation . 

2.1  The  followi  g  techniques  may  be  used  to  build  o) servability  into  the 
system/module . 

a.  Use  unused  I/O  pins  to  provide  access  to  internal  nodes 
otherwise  unavailable. 

b.  Use  a  parity  generator  to  achieve  high  observability  on 
digital  PCB 1 s  without  excessive  reliance  on  board  edge 
connector  pins  as  test  points. 

c.  Test  Points 

Select  test  point  locations  for  maximum  access  to  buried 
nodes.  Strategic  placement  of  test  points  is  far  more 
important  than  quantity. 

Critical  locations  for  test  points  are: 

-  In  feedback  loops  for  control  of  important  signals 

-  To  subdivide  counter  chains  and  long  sequential 
logic  paths 

-  At  wired  AND  connections  and  similar  high  ambiguity 
paths 

-  At  points  where  high  fan-out  or  high  fan-in  exists  (Figure  1) 

-  On  data  bus  enable  signal  paths 

-  On  BOM  address  lines 

-  On  chip  disable  bus  lines 


Figure  1.  High  Fan-Out /High  Fan-In  Test  Point  Placement 
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-  Between  logic  blocks* 19 ^ 

-  In  circuits  with  hybrid  (combination  digital/analog)  and/or 
redundant  (fault  tolerant)  logic *19* 

Use  display  LED's  on  ths  PCB  to  indicate  proper  operation  of 
important  circuits.  Examples  are:  power  supply  voltage  is 
OK,  clock  is  present,  or  a  phase- locked- loop  is  locked. 

Use  positive  indication  fault  indicators  and  displays  such  that 
a  good  test  always  results  in  an  ON  condition.  A  defective 
indicator  or  display  than  always  indicates  a  fault  condition, 
either  due  to  the  input  or  of  itself  (it  is  faulty) . 

For  a  critical  display,  provide  an  alternate  method  of  testing 
such  as  push-to-test  to  provide  positive  verification  of  its 
operation. 

Use  a  multiplexer  to  reduce  the  number  of  edge-connector  out¬ 
puts  for  fault  isolation,  adjustments,  and  test  points.  ' 

For  the  situation  where  wired  AND  and  Wired  OR  Connections  are 
planned: *1®) 

Avoid  the  use  of  wired  AND  or  OR  because  of  the  ambiguity  that 
is  created  in  trying  to  localize  a  fault  to  the  specific  faulty 
gate.  Do  not  use  wired  AND (OR)  with  high  powered  TTL  or 
Schottky  TTL?  they  cannot  be  direct  driven  low  safely  for  more 
than  1  second. 

Break  wired  and  connections  into  smaller  ambiguity  groups. 
(Figure  2) 


Figure  2.  Redesign  Gate  Network*  to  Reduce  Fault  Ambiguity  Factor 


on  >  o • > 


i.  For  the  case  where  redundant  circuitry  is  used: 

Avoid  logically  redundant  circuits.  A  connection  in  a  circuit 
is  said  to  be  redundart  if  no  change  occurs  in  the  output  of 
the  circuit  when  the  connection  is  open. 


Figure  3.  Provide  Test  Access  by  Redesign  to  Avoid  Fatdt  Redundancy 


Both  circuits  in  Figure  3  generate  the  same  logic  function. 
However,  circuit  (a)  gives  the  same  output  even  when  the  B 
input  on  gate  G1  is  stuck  at  1,  or  the  C  input  on  gate  G2 
is  stuck  at  1.  Zn  either  case,  the  output  becomes  F  *  ABC+AD, 
hence  the  fault  cannot  be  isolated  to  a  specific  gate. 

With  either  B  or  C  stuck  at  1,  circuit  (b)  output  also  becomes 
F  “  ABC+AD,  but  now  the  fault  can  be  isolated  to  the  B*C  gate. 


2.2  One  method  of  presenting  fault  isolation  requirements  is  to  relate  the 
degree  of  isolation  desired  to  component  density. For  the  most  part 
the  ratio  between  component  density  and  fault  isolation  is  relatively 
constant.  Figure  4  represents  digital  PCB's.  Isolation  requirements 
presented  in  this  manner  do  not  allow  design  latitudes.  One  method  of 
providing  latitude  is  a  non-compliance  report.  The  key  is  that  the 
responsibility  for  providing  a  testable  design  is  placed  with  the 
designer. 
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IC»  PLUS  3  DISCRETES  IN  100% 
DETECTED  FAULTS 

ICt  PLUS  2  DISCRETES  IN  90% 
DETECTED  FAULTS 

ICt  PLUS  1  DISCRETES  IN  80% 
DETECTED  FAULTS 


Figure  4.  PCB  Level  Component  Isolation  for  Digital  Circuits 
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Quality  Criteria. 

a.  Validation  Phase  Function  V4 , "Analyze  Inherent  Testability  of  Prelimin¬ 
ary  Design"  contains  synopses  of  the  Figures  of  Merit  used  for  Test¬ 
ability.  The  FOMs  must  be  adapted  and  tailored  for  use  in  specific 
programs. 


Some  examples  of  the  Figures  of  Merit  that  may  be  adapted  to  measure 
observability  are: 


Shop  Non-Ambiguity  Ratio  ^1)  = 
(LRU  Testing)* 


Number  of  SRU's  isolated  directly 
without  ambiguity 
Total  number  of  SRU's  in  the  LRU 


Test  Thoroughness 


Amount  of  system/equipment  tested  by 
PIT/ETE 

/Amount  of  system/ \  +  /Amount  of  system/ 
I  equipment  tested  \  (  equipment  not 
l  by  BIT/ETE  J  l  tested  by  BIT/ 

'  /  \  ETE 


) 


Fraction  of  Faults 
Isolated 


Quantity  of  detected  faults  isolated 

_ with  BIT/ETE _ 

Quantity  of  faults  detected 


b.  Other  possible  measures  are: 

Number  of  components  between  test  points 

Sensitivity  of  outputs  with  respect  to  internal  changes 

Ratio  of  inputs  to  components 

Ratio  of  inputs  to  outputs 

Uniformity  of  mapping  inputs  to  outputs,  etc. 


*  LRU  is  a  generic  term  that  may  be  defined  in  terms  of  both  Avionic 
equipment  and  Ground  Systems  equipment. 

-  For  Avionic  equipment  or  systems,  it  is  Line  Replaceable  Unit. 

-  For  Ground  Systems  equipment,  it  is  Lowest  Replaceable  unit. 

LRU  is  defined  as  a  unit  which  is  designated  by  the  plan  for  maintenance 
to  be  removed  upon  failure  from  a  larger  entity  (equipment,  system)  in 
the  letter's  operational  environment.  (MXL-STD-1309-B) 

A  LRU  is  composed  entirely  of  SRUs.  A  SRU  is  defined  as  follows: 

SBU  (Shop  Replaceable  Unit)-  A  generic  term  which  includes  all  the 
packages  within  a  LRU,  including  chassis  and  wiring  as  a  unit. 
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c.  A  methodology  was  developed  In  reference.  (22)  that  evaluates  the 
testability  merits  of  a  printed  circuit  board  (PCB)  through  a 
"Figure  of  Merit"  rating  system  that  weighs  the  "difficult  t^  test" 
and  "easy  to  test"  aspects  of  a  circuit  design.  Reference  (22)  is 
condensed  as  an  appendix  to  Task  V4B . 


3.  Completion  Criteria. 

The  task  may  be  considered  complete  upon  acceptance  at  the  PDR  of  the 
observability  features  in  the  design. 


i 


PHASE: 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  \2H~ 

VALIDATION 


FUNCTION:  System  Design,  Testability  Incorporation 

TASK  TIT**1-  Determine  the  number  and  placement  of  UUT  test  points 


TASK  OBJECTIVE:  Provide  design  guidance  for  optimum  placement  of  test  points 

at  all  system/ equipment  indenture  levels  for  fault  detection, 
fault  isolation  and  inplace  calibration  and  alignment. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE  INPUT  TO  T  _  OUTPUT  FROM  T 


•  Design  e  Test  Point  Selection  •  Qualitative  Testability 

Engineering  Requirements  (Initializa¬ 

tion,  Controllability,  & 
Observability) 


COST  TRADEOFF  INTER-REIATIONSHIPS:  Optimized  placement  of  test  points 

reduces  diagnostic  software  development 
costs  and  reduces  test  resource  support  costs  in  the  deployment  and  operations 
phase. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  proper  placement  of  test  points  is  consistent  with  good  maintenance 
practices.  Test  points  should  provide  for  validity  and  simplicity  in 
the  detection  and  isolation  of  faults  in  addition  to  providing  for  evalua¬ 
tion  of  the  functional  performance  status  of  the  prime  system.  With 
proper  test  point  placement,  faults  can  be  isolated  to  those  assemblies 
or  components  whose  alignment,  repair,  removal/replacement  or  other 
corrective  action  is  feasible  and  consistent  with  the  maintenance  and 
support  planning. 


Controllability  (Task  V2F)  and  Observability  (Task  V2G)  are  dependent  on 
test  point  arrangement  and  should  be  reviewed  and  considered  in  conduct¬ 
ing  this  task  of  test  point  placement. 


Test  point  placement  needs  to  be  treated  at  each  testable  level  of  the 
prime  system  hierarchical  indenture,  i.e.,  the  system  itself  represents 
a  Unit  Under  Test  as  does  eve -y  lower  level  UUT. 


2.  Implementation. 

2.1  Testability  is  most  effectively  instilled  into  a  system  when  design 

guidelines  are  provided  as  design  commences.  It  is  inefficient  for  the 
testability  {or  any  other  -ility)  engineer  to  take  the  role  of  a  post¬ 
design  critic  and  difficult  as  well  as  uneconomical  for  designers  to 
incorporate  changes  to  documented  designs  when  those  changes  reflect 
design  criteria  which  could  have  and  should  have  been  disclosed  during 
or  ahead  of  design  formulation.  Timeliness  is  therefore  a  principal 
responsibility  for  the  testability  engineer,  and  in  all  interfaces,  the 
testability  engineer  should  be  prompt,  active  and  assertive. 


2.2  There  is  much  literature  extant  relative  to  testing  of  digital  circuitry, 
while  analog  testing  is  by  inference  left  to  textbook  methods.  The 
techniques  presented  here  have  been  largely  kept  in  the  form  in  which 
extracted  from  the  source,  hence  they  explicitly  provide  mostly  for 
digital  circuitry.  Analog  applications  may  however  be  readily  inferred. 


2.3  Guidance  for  test  point  placement  follows  in  three  areas:  qualitative 
design  recommendations,  samples  of  test  point  locations  within  wiring 
schemes,  and  figure-of-merit  concepts  to  guide  the  depth  of  assessment 
of  effectiveness  of  test  point  placement. 

a.  Qualitative  design  recommendations  are: 

(1)  Test  points  should  be  selected  based  upon  failure  isolation 
requirements  and  unit  failure  rates. 

(2)  Test  points  selected  should  be  readily  accessible  for 
connection  to  ATE/ETE  via  operational  connectors  or  test 
connectors . 

(3)  Test  points  should  be  chosen  so  that  high  voltage  and 
current  measurements  are  consistent  with  safety  requirements 
of  MIL-STD-454. 

(4)  Test  point  measurements  should  relate  to  a  common  equipment 
ground  which  can  also  be  grounded  at  the  ATE/ETE. 

(5)  Test  points  should  be  decoupled  from  the  ATE/ETE  to  assure 
that  degradation  of  equipment  performance  does  not  occur  as 
a  result  of  connections  to  the  ATE/ETE. 
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(6)  Test  points  of  high  voltage/currant  should  be  physically 
Isolated  from  test  points  of  low  logic  level  signals. 

(7)  Test  points  for  in-place  calibration/alignment  should  be 
provided  as  required. 

(8)  Test  poll's  should  be  selected  for  maximum  functional 
observability  (see  Task  Reference  Number  V2G)  and  con¬ 
trollability  (see  Task  V2F) . 

(9)  Test  points  should  be  located  at  assembly  level  such  that 
fault  isolation  to  a  module  can  be  achieved  without  use  of 
me lule  test  points  or  disassembly  of  the  assembly. 


b.  Samples  of  test  point  locations  within  wiring  schemes  are: 

(1)  Use  wired  AND  (OR)  for  external  control  connections  as  a 
means  of  injecting  test  stimuli  into  card  under  test .(10) 
(See  Figure  1.) 


I/O  PIN  OR 
TEST  POINT 


Figure  1.  Wired  OR  for  TEst  Control  Input 


(2)  Select  test  point  locations  for  maximum  access  to  buried 
nodes.  Strategic  placement  of  test  points  is  far  more 
important  than  quantity. 

(3)  Place  test  points  between  logic  blocks,  in  circuits  with 
hybrid  (combination  digital/analog),  in  redundant  (fault 
tolerant)  circuitry,  to  monitor  secondary  d.c.  power  supplies 
and  for  performance  monitoring  of  prime  equipment  interface 
signals. 
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(4)  Important  locations  for  test  points  are:^°) 

e  In  feedback  loops  for  control  of  important  signals 

e  To  subdivide  counter  chains  and  long  sequential 
logic  paths 

e  At  wired  AND  connections  and  similar  high 
ambiguity  paths 

e  At  any  point  where  high  fan-out  or  high  fan-in  exists 
(see  Figure  2) 

e  On  data  bus  enable  signal  paths 

e  On  SOM  address  lines 

e  On  chip  disable  bus  lines 


FIGURE  2.  HIGH  FAN-OUT/HIGH  FAN-IN  TEST  POINT  PLACEMENT 


•  f[. 


Figure  of  Merit  concepts  for  test  point  placement  effectiveness  are: 

(1)  The  numbers  and  location  of  the  system/equipment  test  points 
should  be  determined  to  provide  maximum  functional  controllability 
(task  reference  number  V2F)  and  observability  (task  reference 
number  V2G) .  The  Figures  of  Merit  fc  *  controllability  and 
observability  may  be  used  as  indicators  for  test  point  optimiza¬ 
tion. 

(2)  The  test  point  number  and  location  should  be  such  that: 

(a)  the  organizational  level  test  equipment  (BITE  or  ETE)  can 
fault  isolate  the  system/equipment  to  the  replaceable 
assembly  {subsystem/module)  level  without  system/equipment 
disassembly. 

(b)  the  intermediate  level  test  equipment  (BITE  or  ETE)  can 
fault  isolate  the  organizational  level  replaceable  assembly 
to  the  intermediate  level  replaceable  module  (e.g. ,  PCB/IC) 
without  disassembly  of  the  assembly. 

(c)  the  depot  level  test  equipment  (ETE  or  ATE)  can  faul'; 
isolate  the  intermediate  level  module  to  a  replaceable 
component  (e.g.,  PCB/IC)  without  disassembly  of  the  module. 

(3)  Figures-of-merit  concepts  that  can  apply  to  test  point/ 
quantities  are: 

(a)  FOM  =  number  of  test  points 
number  of  circuits 


where  both  parameters  are  defined  at  the  same 
UUT  test  level. 

(b)  FOM  ■  Sum  of  failure  rates  in  circuits  with  test  points 
Sum  of  failure  rates  of  all  circuits 


(For  the  first  FOM,  the  literature  does  not  suggest  a 
method  for  counting  the  number  of  circuits.  A  suitable 
quantification  may  be  tailored  to  particular  applica¬ 
tions  as  sene  function  of  inputs  and  outputs  (countable) 
for  a  physical  entity,  e.g.,  a  subsystem,  a  module  or 
circuit  card  or  a  multi- lead  consonant.) 
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(4)  These  FOM  are  essentially  conceptual  as  currently  defined 
and  no  ranges  of  values,  either  possible  or  desirable, 
have  been  established  as  standards. 


sletion  Criteria. 


The  task  of  UUT  test  point  determination  is  confuted  when  the  design 
review  process  actions  have  been  completed  and  the  government  accepts 
the  test  point  arrangement  as  designed. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V2X 

VALIDATION 


System  Design/  Testability  Incorporation 
Incorporate  system  J.evel  BIT 

Assure  that  the  design  provides  system  level  BIT  for  (1) 
Performance  Monitoring,  (2)  Initial  Fault  Detection  and 
Initial  Fault  Isolation  to  a  major  subsystem  or  equipment,  and  (3)  System 
Verification  following  corrective  maintenance  or  reconfiguration  to  a 
degraded  mode. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 
Engineering 

e  Boundary  Between  BIT/ETE 

e  BIT/ATE/ETE  Tradeoff 
Analysis 

e  Design 

Engineering 

e  Hardware  BIT  Features 

e  HW  BIT  Tradeoff  Analysis 

e  Application 
Software 

e  Test  Program  Architecture 
and  Listings 

e  SW  BIT  Tradeoff  Ahalysis 

COST  TRADEOFF 

INTER-RELATIONSHIPS:  Well  designed  BIT  improves  operational 

availability,  performance,  and  a ^portability.  The  costs  of  all  test  and 
repair  resources  can  be  substantially  reduced  as  a  result. 


PHASE: 

FUNCTION: 

TASK  TITLE: 

TASK  OBJECTIVE: 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  following  requirements  should  be  applied  to  the  design: 
a.  Perfonnance/test  requirements  tradeoffs. (5) 

Tradeoff  requirements  for  prime  equipment  performance  and  requirements 
for  built-in-test  and  monitoring.  As  an  example,  a  safety  margin 
built  into  a  prime  equipment's  output  signal  specification  might  be 
relaxed,  with  an  attendant  cost  savings,  if  the  accuracy  and  depend¬ 
ability  of  the  monitoring  circuits  could  be  improved. 
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b.  Manual/automatic  teat  tradeoffs. 

Decisions  regarding  the  type  of  test  equipment  to  be  used  for  system 
monitoring  and  maintenance  should  be  made  based  upon  repair  policies 
and  overall  maintenance  plans.  Tradeoffs  should  be  made  for  test 
requirements  at  each  maintenance  level ,  considering  test  complexity, 
time  to  fault  isolate,  operational  environment,  logistic  support 
requirements,  development  time  and  cost.  The  degree  of  testing 
automation  should  be  consistent  with  the  planned  skill  levels  of  the 
equipment  operators  and  maintenance  personnel. 

c.  BIT/ATE  Utilization. 

Establish  a  two-level  automatic  test  concept  which  is  driven  by  the 
natural  break  between  BIT  capabilities  and  ATE  capabilities: 

<1)  Built-in  test  should  be  utilized  to  provide  initial  fault 
detection  for  a  system  or  equipment  and  to  provide  initial 
fault  isolation  to  a  small  group  of  modules. 

(2)  off-line  Automatic  Test  Equipment  should  be  used  to  provide 
fault  detection  for  a  module  as  a  Unit  Under  Test  (UUT) 
and  provide  fault  isolation  to  components  within  a  module. 

d.  BIT/ATE  Coordination. 

Plan  for  maximum  utilization  of  available  BIT  capability  within  a 
UUT  in  determining  off-line  test  requirements. 

e.  BIT/portable  tester/central  shop  tradeoffs. 

Perform  tradeoffs  to  determine  the  optimum  mix  of  built-in  test 
equipment,  portable  testers,  and  centralized  maintenance  shops  to 
support  organizational  maintenance. 

f .  Functional  partitioning  for  BIT. 

Design  the  equipment  such  that  relatively  small,  independent,  and 
manageable  blocks  of  circuitry  can  be  defined  as  the  basis  of  test 
derivation,  test  documentation  and  test  evaluation.  j 

g.  System-level  BIT.  j 

Incorporate  suitable  built-in  test  features  into  the  system  to  J 

provide  initial  failure  detection  and  to  provide  initial  failure  1 

localization  to  a  major  subsystem  or  equipment.  | 

h.  System  verification. 

Utilize  built-in  test  in  verifying  the  operability  of  the  system 
following  a  corrective  maintenance  action  or  a  reconfiguration  J 

into  a  degraded  mode.  j 


i.  Performance  monitoring. 

Coordinate  performance  monitoring  requirements  with  BIT  requirements 
to  make  optimum  use  of  resources,  as  required  by  MIL-STD-1326. 

j.  Planned  maintenance. 

Make  maximum  use  of  available  built-in  test  hardware  and  software 
in  developing  planned  maintenance  procedures. 

k.  Built-in  test  tradeoffs. 

Incorporate  a  mix  of  built-in  test,  external  automatic  test  and 
manual  test  which  provides  the  highest  level  of  failure  resolution 
consistent  with  operational  availability  requirements  and  life  cycle 
cost  requirements.  Alternate  designs  should  be  analysed  and  traded 
off  against  requirements  of  performance,  supportability,  and  cost 
to  arrive  at  a  configuration  best  meeting  the  requirements  at 
minimum  cost.  General  guidance  in  this  area  may  be  found  in 
NAVMATINST  3960.9. 

l.  False  Alarms. 

A  false  alarm  is  defined  as  an  indicated  failure  where  no  failure 
exists.  The  system  level  BIT  should  have  a  false  alarm  rate 
(frequency  of  occurrence  of  false  alarms)  of  less  than  5%  of  the 
system  failure  rate.  Furthermore,  the  false  alarms  should  occur 
only  as  a  result  of  a  failure  within  the  system  level  BIT  equipment. 


2.  Implementation . 

2.1  The  following  guidelines  mt.y  be  used  to  incorporate  system  level  BIT 

into  the  system  design. 

a.  BIT  includes  checks  made  by  operational  software  such  as  reason¬ 
ableness  checks  and  calibration  checks.  It  includes  continuous 
monitoring  of  integrity  of  data  transmissions  using  parity  and 
other  codes,  and  timing  of  data  transfers  and  other  operations. 

It  includes  the  use  of  end-around  loop  tests,  microdiagnostics  in 
control  BOM,  redundancy  within  basic  control  logic,  and  redundancy 
in  data  paths  for  improved  fault  isolation. (23) 

b.  BIT  may  be  either  operator  or  self-initiated  and  may  display  an 
indication  visible  to  the  operator  or  result  in  an  indication 
detected  by  system  confidence  detection  and  control  circuits.  (23) 

c.  Incorporation  of  self-test  capability  into  PCBs  composed  almost 
entirely  of  large  scale  ICs  is  a  viable  and  powerful  alternative  to 
testing  solely  controlled  by  external  test  equipment.  Use  of  self¬ 
test  macro  sequences  initiated  from  and  controlled  by  the  STE  program, 
combines  the  uniqueness  of  a  dedicated  self-test  with  the  flexibility 
of  an  externally  generated  test. 


Anon?  the  advantages  that  can  accrue  with  an  internal  self-test  are 
(1)  operator  intervention  is  bypassed,  (2)  a  fault  is  isolated  as 
soon  as  it  occurs,  (3)  a  fault  can  be  isolated  to  a  replaceable 
component,  and  (4)  system  downtime  is  minimized. 

Maintenance  aids  to  incorporate  into  the  design  include  built-in 
product-initiated  self-tests  and  user-callable  test.  Self-initiated 
routines  are  automatically  exercised  by  the  product  everytime  same 
preset  condition  is  met.  A  user-callable  routine  is  exercised  only 
when  the  serviceman  selects  it.^O) 


2.2  A  comparison  of  PCB  test  system  characteristics  using  five  different 
test  scenarios  is  illustrated  in  Figure  1.(10)  The  figure  illustrates 
BIT  advantages. 

First  is  built-in  test,  which  subdivides  into  hardware  BIT,  software 
BIT,  and  the  selected  combination  of  them,  called  microdiagnostics. 
Second  is  drive/sense  (bed-of-)  nails.  Third  is  sense  only  nails 
stimulated  by  edge  (I/O  pin)  drive.  Fourth  is  edge  drive/sense. 

Fifth  is  scan/set  which  provides  a  data  path  out  of  the  card  via  added 
logic  and  output  pins  separate  from  the  functional  logic;  the  purpose 
is  to  expose  internal  nodes  of  a  chip  or  PCB  for  test  purposes.  The 
two  common  forms  of  scan  are  bit-serial  using  a  shift  register,  and 
multiplex.  The  shift  register  form  allows  both  scan  of  results  and 
set(ing)  of  input  vectors. 


2.3  BIT  actual  failure  rates  have  been  found  to  exceed  the  rates  predicted 
from  handbooks.  (24)  The  ratio  is  1.25  actual  to  every  predicted  BIT 
failure. 


2.4  System  level  BIT  should  meet  the  following  quantitative  criteria  unless 

otherwise  specified. 

a.  Fault  Detection  -  System  level  BIT  should  detect  95*  of  all  system 
failures. 

b.  Fault  Location  -  System  level  BIT  should  isolate  90%  of  all 
detected  failures  to  a  single  subsystem  (assembly) ,  95%  of  all 
detected  failures  to  no  more  than  two  subsystems,  and  100%  of 
detected  failures  to  no  mors  than  three  subsystems. 

c.  False  Alarm  -  System  level  BIT  should  have  a  false  alarm  rate 
of  less  than  5%  of  the  system  failure  rate. 
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TEST  SCENARIO 


TEST 

SYSTEM 

CHARACTERISTICS 

— 

DESIGN  PCS  FOR 
BIT  USING 
MICRODIAGNOSTICS 

DRIVE/SENSE 

BED-OP-NAILS 

EDGE  PIN  STIMULUS 
COMBINED  WITH 
BED-OF-NAILS 
SENSE  ONLY 

STIMULUS  AND 

SENSE  AT  EDGE 

FINS  ONLY 

EDGE  PIN  WITH 
PCB  DESIGNED 

FOR  SCAN/SET 

TEST  TUB  REQUIRED 

FEU  SECONDS 

FEW  MINUTES 

FEW  SECONDS 

FEW  SECONDS 

FEW  SECONDS 

FAULT  RESOLUTION. 
FAULT  RESOLVED  TO- 
CONCNTS 

VERY  HIGH. 

NORMALLY  TO 

FAILINC  CHIP- 
ADDITIONAL  CONTROL 
NEEDED  TO  RESOLVE 
CHIPS  ON  A  BUS 

VERY  HIGH  RESO¬ 
LUTION  OF  CHIP 
FAILURES. 

FOIL  NOT 

CHECKED  WELL- 
PRETEST  PBW 

AS  BACKUP 

HIGH  RESOLUTION. 

TO  CHIP  OR  BUS  NODE- 
ALSO  CHECKS  FOIL. 

LOW  RESOLUTION. 
TO  EXTERNAL 
NODE- 

HIGH  RESOLUTION. 
TO  CHIP  OR  BUS 
NODE- 

RELIABILITY  OF 
TESTER 

NO  TESTER  REQUIRED. 
MUST  HAVE  PROCESSOR 
ON  PCB  TO  RUN  BIT 

LOW  MODERATE 

HICH  MODERATE 

— 

HIGH 

...  ^ 

HIGH 

SOFTWARE  DEVELOF- 
tONT  COST 

NOMINAL 

NOMINAL 

HIGH 

HIGH 

HIGH 

TEST  GENERATION 
COST/MANPOWER  AND 
COMPUTER  TIME. 

NOMINAL 

NOMINAL 

HIGH 

VERY  HIGH 

HIGH 

EASE  OF  TEST 

UPDATE 

EXCELLENT .  ONLY 
NEEDED  TO  MODIFY 
BUILT-IN  DIAG¬ 
NOSTIC  PROGRAM 

GOOD.  INTER¬ 
CONNECTION 
BETWEEN  CHIPS 

IS  OF  MINOR 
IMPORTANCE . 

FOR  NEW  CHIPS 
ADD  A  NEW 
FUNCTIONAL 

TEST  TO 

EXISTING 

PACKAGE 

— 

POOR.  EDGE  PINS  USU 
ANY  PART  OF  THE  CARD 

ALLY  REQUIRE  A  WV 
IS  CHANGED. 

10LE  NEW  TEST  IF 

COST  TO  REPRODUCE 
THE  TESTER 

NONE.  NO  TESTER 
REQUIRED 

VERY  HIGH 

HIGH 

NOMINAL 

NOMINAL 

TEST  DATA  BASE 
SIZE. 

COST 

COMMENTS 

SMALL. 

LOW,  VERY  INFRE¬ 
QUENT  UPDATES. 

EASY  TO  DO  USING 
MICRODXAGNOST ICS 

SMALL  TO 
MODERATE. 
NOMINAL,  FRE¬ 
QUENT  UPDATES. 
ONE  TEST  PER 
CHIP  TYPE 

LA 

HI 

FR 

RGE. 

SH. 

F.QUF.NT  UPDATES. 

DESIGN  GUIDELINES 
AND  CONSTRAINTS 

ON  PC RUT  (IMPACT 

ON  COST  AND/OR 

PERFORMANCE 

(C/P) 

MUST  INCLUDE  BIT 

IN  HARDWARE  AND 
SOFTWARE.  SEVERE 
C/P  IMPACT. 

VERY  STRONG 
CONSTRAINTS 

BOARD  LAYOUT 

AND  PARTS 
LOCATION 
CONSTRAINTS. 
SLIGHT  C/P 
IMPACT. 

MODERATE 

CONSTRAINTS 

BOARD  LAYOUT  AND 
PARTS  LOCATION 
CONSTRAINTS. 

SLIGHT  C/P 

IMPACT. 

LOW  MODERATE 
CONSTRAINTS 

C/P  IMPACT  MAY 
BECOME  PROHIBI¬ 
TIVE  IN  LARGE 

LSI . 

MILD 

CONSTRAINTS 

SPECIFIC  CON¬ 
STRAINTS  TO 
INCORPORATE 

SCAN/ SET  INTO 
DESIGN . 

NOMINAL  C/P 

IMPACT. 

STRONG 

CONSTRAINTS 

ABILITY  TO  TEST 

AT  HIGH  SPEED 

HIGH 

HIGH 

NOMINAL 

NOMINAL 

NOMINAL 

ABILITY  TO  TEST 
CARDS  CONTAINING 
LSI,  VLSI,  WLS1 

HICH 

MODERATELY 

HIGH 

LOW 

VERY  LOW 

MODERATE  IF 
SCAN/SET  IS  ON 

PCB  ONLY. 

HICH  IF  SCAN/SET 
IS  BUILT  INTO  LSI 

CONSTRAINTS  ON 

TEST  WTHODOLOCY/ 
TEST  PROCEDURE 
DEVELOPMENT 

VERITY  THE 

KERNEL,  THEN 

EXPAND  ON  IT 

BASICALLY  A  SET 
OF  CHIP  TEST 
ROUTINES 

NF ED  A  MEANS  OF  AUTO 
PACTION  OF  RESULTS-D 
QUANTITY  OF  LSI 

TEST  VECTOR  GENS 
VTA  FOR  PCB*S  WIT 

_ 

RATION  AND  COM- 
H  SIGNIFICANT 

Figure  1.  A  Comparison  of  Test  System  Characteristics  Using  Five  Different  Test  Scenarios 
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d.  System  MTBF  -  Any  reduction  in  system  MTBF  should  be  offset  by 
an  increase  in  system  maintainability  (reduced  MTTR) . 

The  ratio  MTTR  for  a  system  with  BIT  should  be  less  than  the 
MTBF 

same  ratio  for  an  equivalent  system  without  system  level  BIT. 

standard  Figures  of  Merit  may  be  used  as  measures  for  system  level  BIT.  See 
Validation  Phase,  Task  Function  V4. 


3.  Completion  Criteria. 

The  task  is  considered  couplets , upon  government  acceptance  of  the 
development  specifications. 


m 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V3A 


PHASE: 

VALIDATION 

FUNCTION: 

BIT  Tradeoffs 

TASK  TITLE: 

Tradeoff  alternate  designs  of  BIT,  external  automatic 
test  and  manual  test  mixes 

TASK  OBJECTIVE: 

Provide  for  optimized  mixes 
levels  of  test,  maintenance 

of  BIT/BITE/ETE  at  all 
and  repair. 

DESIGN  DISCIPLINE 

TESTABILITY  (T)  INTER-REIATIONSHIPS : 

T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 
Engineering 

e  System  Level  BIT/ETE 

e  Qualitative  T 
Requirements 

e  Design 

Engineering 

e  Performance,  HW  BIT 

e  Qualitative  T 
Requirements 

e  Maintainability 
Engineering 

e  Maintenance  Concept 

•  Merge  T  and  M 
Requirements 

e  Support 
Equipment 

e  Selected  or  Available  ETE 
New  ETE  Analysis 

vs  e  Qualitative  T 
Requirements 

e  Application 
Software 

e  Test  Sequences,  SW  BIT 

e  Qualitative  T 
Requirements 

e  Life  Cycle 

Cost 

e  Cost  Data  Guidance 

•  Test  Cost  Data 

e  ILS  Engineer¬ 
ing 

e  Repair  vs  Discard  Concepts  e  Test  Equipment  Options 

and  Information 

e  Program  Manager 

•  Testability  Analysis 

COST  TRADEOFF  INTER-RELATIONSHIPS  8  The  selection  of  the  mix  of  test  methods 

to  support  the  prime  system  has  major  impacts  on  costs  of  acquisition  and  own¬ 
ership.  Non-recurring  costs  affected  include  prime  systest  design,  software, 
development  test  interface  designs,  technical  data  and  training.  Recurring 
acquisition  costs  affected  include  prime  system,  test  equipment  and  test 
program  sets  (interface  devices)  together  with  such  acquisition  logistics  as 
initial  spares.  Operating  and  support  costs  affected  include  operator  and 
maintenance  skill  and  labor  levels  and  replacement  training. 
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TASK  SYNOPSIS 


1.  Task  Requirements . 

The  task  requires  performance  of  iterative  sets  of  tradeoffs. 

1.1  Tradeoffs. (5) 

a.  Built-in-test  tradeoffs.  Trade  off  alternatives  of  built- in-test, 
external  automatic  test,  and  manual  test  to  arrive  at  the  optimum 
operational  level  test  equipment  mix. 

b.  BIT/Portable  Tester /Central  Shop  Tradeoff.  Trade  off  alternatives 
of  BIT,  portable  testers,  and  centralized  maintenance  shops  to  arrive 
at  the  optimum  organizatic'  al  level  test  equipment  mix.  (In  many 
appl  nations,  operational  and  organizational  level  testing  are 
unified. ) 

c.  Manual/Automatic  Test  Tradeoffs.  Trade  off  alternatives  of  manual 
or  automatic  test  equipment  to  arrive  at  the  optimum  test  and  test 
equipment  operator  requirements  and  personnel  skill  level  matchups 
for  intermediate  and  depot  level  testing.  Interface  with  level  of 
repair  (repair  vs  discard)  analyses. 

d.  Available  Inventory.  A  search  should  be  conducted  to  identify 
applicable  test  equipment  within  the  inventory  or  that  will  be  within 
the  inventory  when  the  system  enters  production  stage.  Adapting 
existing  resources  for  use  with  the  system  is  a  major  step  in  the 
tradeoff  analysis. 


2. 


Implementation . 


2.1  Fundamental  Considerations . 

a.  For  operational  level  testing  there  are  two  basic  test  alternatives, 
on-line  (includes  BIT/BITE)  and  off-line.  Test  Equipment  approaches 
utilizing  the  alternatives  should  be  determined,  examined,  and  a 
selection  of  the  best  alternative  made.  Considerations  to  be  used 
in  selection  of  the  ETE/ATE  alternative  are  cost,  budget,  schedule, 
technical  performance,  and  ability  to  acquire  suitable  management 
capability  for  follow-on  support.  Other  factors  that  should  also  be 
used  as  criteria  are  operational/deployment  considerations,  life 
cycle  cost  impacts,  the  system  maintenance  concept,  development  lead 
times  to  meet  operational  deployment,  economics,  reliability,  main¬ 
tainability,  supportability,  complexity  of  the  test  system  including 
personnel  skill  levels,  system  maturity,  useful  life  requirements, 
anticipated  work  load,  UUT  diversity  and  testability,  and  time 
constraints. (25) 
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b.  For  ground  based  electronics  systems,  offline  organizational  level 
maintenance  and  testing  may  supplement  operational  level  testing 
while  in  most  airborne  electronics  scenarios,  there  is  no  sharp  dis¬ 
tinction  between  operational  and  organizational  level  testing.  Where 
required,  the  same  analysis  as  indicated  in  2.1a  above  may  be  per¬ 
formed  to  provide  for  selection  of  resources.  The  tradeoffs 
considered  include  portable  test  equipment,  and  iteratively  treat 
the  candidate  alternatives  previously  considered  for  operational 
level  testing. 

c.  Intermediate  and  depot  level  testing  exist  primarily  to  provide 
spares  certified  ready  for  use  to  the  operational/organizational 
levels.  Consequently,  these  levels  operate  under  time  and  avail¬ 
ability  constraints  differing  from  those  imposed  on  operational/ 
organizational  levels,  and  most  generally  seek  to  optimize  throughput 
or  production  rites.  While  therefore  looking  to  ATE  as  an  attractive 
alternative,  tradeoffs  at  this  level  must  also  treat  the  candidates 
selected  at  the  lower  echelons,  to  optimally  exploit  such  features  as 
BIT  and  also  to  recognize  tolerance  requirements  (depot  level  test 
tolerances  must  be  as  tight  as  or  tighter  than  intermediate,  which 
must  be  as  tight  as  or  tighter  than  organizational/operational  level 
test  tolerances) . 

d.  The  results  of  the  tradeoff  analyses  conducted  during  the  conceptual 
phase  (Function  C2) should  be  used  in  the  performance  of  this  task. 
Task  C2A,  which  defined  the  BIT/BITE/ETE  interfaces,  is  particularly 
applicable. 

2.2  BIT  and  ETE/ATE  Tradeoff  Factors. 

For  gperational/organizational  level  testing,  some  hybrid  combination  of 

BIT  and  ETE  may  often  be  chosen  so  that  the  test  system  is  described  as 

"BIT-dominant"  or  "ETE -dominant"  rather  than  as  a  pure  BIT  or  pure  ETE. 

The  advantages  of  each  type  are  listed  below:  (26) 

(NOTE:  The  reference  treated  BIT  and  ATE,  adapted  here  to  the  more 
general  concept  of  BIT  and  ETE . ) 


BIT  DOMINANCE 


•  Permits  instantaneous  performance  monitoring. 

•  Eases  burden  on  operator. 

e  Permits  fast,  positive  fault  isolation, 
e  Reduces  shop  testing  time,  especially  test  machine 
occupation  time. 

•  Reduces  shop  instrumentation  requirements. 

•  Can  provide  specialized  built-in-test  capability, 
e  Reduces  shop  skill  level  requirements. 

e  Reduces  shop  manpower  requirements. 

•  Reduces  instruction  manual  and  ETE  software  needs. 

e  Reduces  interface  cabling  between  UUT  and  ETE  equipment. 


•  Reduces  recycling  time  to  to  ETE-undetectable 
operational  failures, 
e  Improves  repeatability. 

e  Avoids  circuit  loading  changes  due  to  probing  and 
prevents  manually  induced  failures, 
e  Permits  probing  of  inaccessable  circuitry, 
e  Reduces  total  life  cycle  cost, 
e  Increases  failure  memory  capability,  if  added, 
e  Permits  prognostics  (failure  prediction) . 
e  Reduces  removal  of  operable  units  (false  alarm  rate) . 


ATE  DOMINANCE 

9  Increases  number  of  parameters  that  can  be  tested, 
e  Detects  faults  missed  by  BIT. 

e  Increases  decision  making  capability  (in  shop) . 

•  Permits  isolation  of  "false  alarm"  causes. 

•  Reduces  initial  hardware  cost  (less  BITE) . 

•  Permits  better  "intermittent"  fault  isolation. 


2.3  Tools  and  Aids  for  Use  in  the  Test  System  Selection  Process. t27* 

In  Table  1,  the  most  applicable  models  and  data  banks  are  listed  by  title, 
function,  applicable  life  cycle  phase,  reference  (source  or  authority) 
and  additional  sources  of  information.  The  life  cycle  column  applies  to 
the  development  or  acquisition  phase.  The  basic  list  was  compiled  in 
1976  and  was  considered  most  applicable  to  the  ATE  selection  process  at 
the  time.  Although  updated  to  some  extent  for  this  synopsis,  it  is  not 
intended  as  a  compendium  of  all  possible  sources  of  aid.  Additional  in¬ 
formation  may  be  obtained  from  the  Test  Equipment  Technical  Support 
Office,  Code  921,  at: 

Naval  Ocean  Systems  Center 

San  Diego,  CA  92152 

Telephone:  Autovon  933-6173 

Commercial  714-225-6173 

2 . 4  Recapitulation  of  Tradeoff  Steps, 
a.  On-Line  Test  System 

(1)  Research  literature  and  data  banks  (Table  1) 

(2)  Analyze  approaches  (selected  test  system/subsystem)  for 
applicability  to  prime  system  requirements 

(3)  Evaluate  alternatives  using  life  cycle  costing  and  system 
performance  requirements  and  select  best  approach 

(4)  Proceed  to  design  of  system  and  test  equipment  (or  modify 
selected  test  equipment) . 


b.  Off-Line  ETE/ATE 


(1)  Search  literature  and  data  banks  (Table  1) . 

(2)  Identify  alternatives  from  existing  inventory  (including 
modifications  of  existing  equipment). 

(3)  Evaluate  alternatives  using  life  cycle  costing  and  system 
performance  requirements. 

(4)  Select  best  approach  and  document  selection. 

2.5  Cost  Analysis  Aids. 

a.  BIT  Life  Cycle  Cost  Tradeoff  Model.  The  BIT  Life  Cycle  Cost  (LCC) 
tradeoff  model  found  in  reference  (28)  provides  cost  delta  informa¬ 
tion  to  assist  system  designers  in  the  selection  of  alternate  BIT 
concepts  and  design  features  through  LCC  tradeoffs.  This  model 
consists  of  five  mathematical  equations,  each  describing  a  simplified 
method  of  computing  an  element  of  LCC  which  is  sensitive  to  the  BIT 
features.  The  costs  calculated  by  the  model  are  not  the  total  (real 
world)  costs  of  the  LCC  elements  but  are  relevant  and  useful  in 
analyzing  the  incremental  cost  impact  of  alternate  designs,  concepts, 
and  test  subsystem  features.  The  model  is  configured  to  use  UUT 
level  performance  and  design  parameters.  The  LCC  impact  (delta)  of 
the  subsystem's  constituent  UTJTs  is  sunned  by  the  model  to  provide 
the  total  subsystem  incremental  cost  benefit  or  penalty. 

The  elements  computed  by  the  five  equations  are: 

o  RDTSE  cost 
o  Acquisition  cost 
o  Operation  and  support  cost 
o  Availability  cost 
o  Flight  penalty  cost 

b.  ATE  Cost  Drivers.  Reference  (29)  provides  analysis  of  economic 
criteria  relevant  to  selection  of  ATE.  The  criteria  were  oriented  to 
factory  and  service  center  operations  and  therefore  have  seme  applica¬ 
tion  to  depot  level  testing  (and  by  inference  to  intermediate  level). 
Table  2  provides  condensed  selection  guidance  factors  for  ATE. 


TABLE  1 


TOOLS  &  AIDS  FOR  USE  IN  THE  TEST  EQUIPMENT  TRADEOFF  AND  SELECTION  PROCESS 


DATA  BANKS 

SOURCE  TITLE;  GIDEP  Metrology  (Formerly,  Secretariat,  Electronic 

Test  Equipment,  Project  SETE) 

TYPE  FUNCTION:  Data  Bank-technical  information,  cost  data,  manufacturer, 

applications,  inventory,  development  and  production  data, 
and  evaluation  data  on  all  types  of  test  equipment,  both 
on-line  and  off-line. 

APPLICABLE  LIFE 

CYCLE  PHASE:  Conceptual,  Advanced  Development,  Full  Scale  Development 

REFERENCES:  NAVMATINST  5200. 35 

MIL-STD  1556 
NAVMAT  Notice  5200 

CONTACT  FOR  (Project  SETE,  Government-Industry  Data  Exchange  Program 

ADDITIONAL  Code  862)  Fleet  Missile  Systems  &  Evaluation  Group, 

INFORMATION:  Corona  CA  91720;  Telephone  AV:  933-4351 


SOURCE  TITLE:  Avionic  Systems  Test  Equipment  Comparator  (ASTEC) 

TYPE  FUNCTION:  Data  Bank-Comparing  and  matching  electronic  UUTs  (pin  by 

pin)  with  off-line  tester  capabilities. 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development,  Full  Scale  Development 

CONTACT  FOR  Commanding  Officer,  Naval  Air  Engineering  Center,  Code 

ADDITIONAL  SE-31,  Philadelphia,  PA 

INFORMATION:  Telephone:  AV  443-4531 


SOURCE  TITLE:  ATE  Data  Bank 

TYPE  FUNCTION:  Data  Bank-Stores  off-line  ATE  characteristics  (includes 

ATE  building  block  information) 

APPLICABLE  LIFE  Advanced  Development,  Full  Scale  Development 
CYCLE  PHASE: 

REFERENCES :  Automatic  Test  Equipment  Acquisition  Planning  Guide  1/31/74 

CONTACT  FOR  Headquarters  San  Antonio  Air  Logistics  Center 

ADDITIONAL  Kelly  Air  Force  Base,  TX  78241 

INFORMATION:  Telephone:  AV  6127 
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TABLE  1 


DATA  BANKS  (continued) 

SOURCE  TITLE:  PCB  Tester  Data  Bank  (Digital  and  Analog) 

TYPE  FUNCTION:  Data  Bank -Compares  the  characteristics  of  PCBs  to  the 

capabilit- es  of  available  military  and  commercial  testers. 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development,  Full  Scale  Development 

REFERENCES ;  RADC  TR-76-106;  RADC  Tk-79-253 

CONTACT  FOR  Mr.  James  Saporito,  RADC/RBET 

ADDITIONAL  Griffiss  AFB  NY  13441 

INFORMATION:  Telephone:  AV  587-4205;  (315)  330-4205 


MODELS  (BIT/BITE/On-Line) 

SOURCE  TITLE:  Built-in  Test  Evaluation  Model  (BITEM) 

TYPE  FUNCTION:  Model -Evaluates  specific  BITE  configurations 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development,  Full  Scale  Development 

REFERENCES:  Final  Report  RCA  Contract  N00123-73-C-1326 

CONTACT  FOR  Naval  Oceans  Systems  Center,  Code  921 

ADDITIONAL  San  Diego  CA  92152 

INFORMATION:  Telephone:  AV  933-6173;  (714)  225-6173 


SOURCE  TITLE:  Determination  of  BIT  Effectiveness  Utilizing  Simulation 

Language  &  General  Purpose  Computer 

TYPE  FUNCTION:  Model-Predicts  BIT  Effectiveness  for  Radar  &  other 

applications . 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development,  Full  Scale  Development 

REFERENCES:  Norden  Report  3676  R  1101 

CONTACT  FOR  Norden  Division  of 

ADDITIONAL  INFO:  United  Aircraft 
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TABLE  1 


MODELS  (continued) 

SOURCE  TITLE:  An  Effectiveness  Study  for  System  Formulation  of 

Centralized  Automatic  Test  System 

TYPE  FUNCTION:  Model-Evaluates  the  effectiveness  of  on-line 

centralized  ATE. 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development 

REFERENCES:  DDC  (AD  844872L) 

CONTACT  FOR  Naval  Ocean  Systems  Denter,  Code  921 

ADDITIONAL  San  Diego,  CA  92152 

INFORMATION:  Telephone:  AV  933-6173;  (714)  225-6173 


Technique  for  Evaluating  Avionics  Built-in  Test 
Model-Methods  for  evaluating  the  effectiveness  of  BIT. 


Advanced  Development,  Full  Scale  Development 

Contract  N00019-71-C-0312 

Naval  Air  Systems  Comnand 
Washington  DC  20361 


SOURCE  TITLE:  A  Model  for  Analysis  for  Value  &  Risk  of  a  Built-in  Test 

Equipment  System 

TYPE  FUNCTION:  Model-Measures  the  effectiveness  for  built-in  test  equipment. 

APPLICABLE  LIFE 

CYCLE  PHASE:  Advanced  Development,  Full  Scale  Development 

REFERENCES :  LD29196 

CONTACT  FOR  Xn-house  study 

ADDITIONAL  INFO:  Capt.  Joseph  Krutulis  USA 


SOURCE  TITLE: 

TYPE  FUNCTION: 

APPLICABLE  LIFE 
CYCLE  PHASE: 

REFERENCES: 

CONTACT  FOR 
ADDITIONAL  INFO : 
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TABLE  1  (continued) 


SOURCE  TITLE: 


TYRE  FUNCTION: 


A  Reference  Library  of  Analytical  Models  and  Simulations 
Which  Can  Be  Used  in  the  ATE  Acquisition 

ATE  Model  Library-  Maintained  to  assist  Navy  Managers  and 
authorized  contractors  in  the  selection  and  application  of 
mathematical  models  relative  to  specific  decision  require¬ 
ments.  Provides  model  redommendations  in  order  of 
applicability;  specify  input  requirements,  describe  output 
interpretation,  and  model  limitations. 


APPLICABLE  LIFE 
CYCLE  PHASE: 


Conceptual,  Advanced  Development,  Full  Scale  Development 


SOURCES  OF  Naval  Weapons  Engineering  Support  Activity  (NAVWESA) 

ADDITIONAL  4204  Maylock  Lane 

INFORMATION :  Fairfax,  VA  22030 
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TABLE  2. 


ATE  ECONOMY  AND  LONGEVITY  FACTORS 


HARDWARE 

o  Make  modular  for: 

-  Commonality 

-  Versatility 

-  Provisioning 

o  Provide  standardized: 

-  Architecture 

-  Bus  structure 

-  Packaging 

-  Human  interface 

o  Design  to  be: 

-  Highly  self -testable 

-  Reliable/maintainable 

-  Raconfigurable 

-  Environmentally  versatile 

o  Develop  ATE  hardware/software 
as  a  total  system 


SOFTWARE 

o  Use  standards 

o  Minimize  hardware/software 
interface  problems 

o  Minimize  use  of  expensive 
development  resources 

o  Provide  responsive/efficient 
development  environment 

o  Recognize  capabilities/ 
limitations  of  developers 

o  Develop  ATE  software/hardware 
as  a  total  system  .  ;  '  v 


2 . 6  Documentation .  I 

i  _ 

The  recommended  selections  should  be  documented,  together  with  sufficient 
data  to  provide  for  audit  to  the  depth  required  to  support  program 
decisions.  Documentation  should  show  the  factors  treated  in  analysis  of 
the  alternatives ,  with  visibility  provided  especially  for  prime  system 
testability,  life  cycle  cost,  teBt  and  repair  times,  skill  levels  and 
utilization  of  existing  test  equipment  designs. 


3.  Completion  Criteria. 

3.1  This  task  is  considered  couplets  upon  selection  of  the  test  methodology 
and  test  equipment  and  its  acceptance  at  the  PDR  (Task  V7a) . 


TESTABILITY  TASK  COMPENDIUM 
fasE  Reference  Number  V4A 


PHASE: 


FUNCTION: 

TASK  TITLE: 


VALIDATION 

Inherent  Testability  Analysis  of  Preliminary  Design 

Assess  the  extent  of  qualitative  testability  included 
in  design. 


TASK  OBJECTIVE:  Provide  a  formal  analysis  of  the  potential  (inherent) 

testability  of  the  preliminarv  design  for  feedback  to 


and  dialogue  with  the  design  engineers. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER- RELATIONSHIPS : 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  System 

e  Partitioning,  system  bit/ate 

e  T  Analysis, 

Continued 

Engineering 

Guidelines 

e  Design 

e  Partitioning,  HW  BIT,  UUT 

e  T  Analysis, 

Continued 

Engineering 

Test  Points,  Initialization, 
Controllability,  Parts 
Selected  for  T,  Observability 

Guidelines 

COST  TRADEOFF  INTER-RELATIONSHIPS :  This  task  considers  all  the  tradeoffs 

appropriate  to  achievement  of  a  high  degree  of  testability. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  Determine  whether  the  following  testability  considerations  have  been 
included  in  the  preliminary  design.  (5) 

a.  The  equipment  is  physically  partitioned  to  support  the  test  process. 

b.  Each  physical  partition  is  functionally  ccepatible  with  planned 
ETE/ATE  resources. 

c.  Conservative  timing  tolerances  and  conservative  signal  tolerances 
are  used  in  the  design  whenever  possible. 
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d.  Regular,  structured  designs  are  used  whenever  possible. 

e.  Sufficient  hardware,  firmware,  and/or  software  is  included  to 
confidently  drive  the  equipment  to  a  known  state  or  condition 
prior  to  running  diagnostic  tests. 

f.  The  design  makes  maximum  use  of  existing  operational  resources 
to  support  self-test  in  a  building  block  fashion. 

g.  The  equipment  contains  controls  and  displays  to  provide  a 
suitable  human  interface  for  test  and  maintenance  actions  per 
MIL-STD- 1472 . 


2.  Implementation . 

2.1  In  all  interfaces,  the  testability  engineer  should  be  prompt,  active  and 
assertive.  Testability  is  most  effectively  instilled  into  a  system  if 
design  guidelines  are  provided  when  design  commences.  It  is  inefficient 
for  the  testability  (or  any  other  -ility)  engineer  to  take  the  role  of 
a  post-design  critic.  It  is  difficult  as  well  as  uneconomical  for 
designers  to  incorporate  changes  to  documented  designs  when  those 
changes  reflect  design  criteria  which  could  have  and  should  have  been 
disclosed  during  or  ahead  of  design  formulation.  Timeliness  is 
therefore  a  principal  responsibility  for  the  testability  engineer.  The 
task  of  analyzing  inherent  testability  will  be  simplified  in  those 
hardware  programs  where  the  testability  engineer  has  been  attentive  to 
the  evolving  design. 


2.2  Inherent  testability  is  defined  as  a  hardware  testability  design  charac¬ 
teristic.  The  accumulated  analyses,  tradeoffs  and  documentation  of  the 
following  tasks  form  the  basis  of  the  inherent  testability  analysis. 

Design  units  to  be  conpatible  with  selected  or  available 
ATE. 

Select  parts  for  testability;  provide  direct  designs  and 
test  sequences. 

Design  units  with  testable  physical  partitioning. 
Incorporate  initialization  into  design. 

Incorporate  controllability  into  design. 

Incorporate  observability  into  design. 

Determine  number  and  placement  of  DDT  test  points. 
Incorporate  system  level  BIT. 

General  T  design  guidance. 


a. 

TKN 

V2B 

b. 

TRN 

V2C 

c. 

TRN 

V2D 

d. 

TRN 

V2E 

e. 

TRN 

V2P 

f. 

TRN 

V2G 

g. 

TRN 

V2H 

h. 

TRN 

V2I 

i. 

TRN 

V2A 
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2.3  Assessment  of  inherent  tastability  continues  through  preparation  of  the 
Testability  Analysis  Report  (Function  V5)  and  program  management/ 
government  scrutiny  in  the  PDR. 


3.  Completion  Criteria. 

3.1  The  success  criteria  used  for  the  above  listed  tasks  also  apply  to  this 
analysis,  although  they  must  be  tailored  and  adapted.  Completion  of 
this  assessment  task  is  signified  by  government-approved  resolution  of 
any  identified  problems  and  by  final  acceptance  of  the  testability 
characteristics  in  the  course  of  FDR. 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V4B 


PHASE t 


VALIDATION 


FUNCTION:  Inherent  Testability  Analysis  of  Preliminary  Design 

TASK  TITLE;  Define  Figure-of-Merit  (FOM)  for  inherent  testability 


TASK  OBJECTIVE:  Establish  Figures  of  Merit  for  use  in  evaluating  and 

controlling  the  testability  inherent  in  the  prime  system 
design, 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS:  None. 


COST  TRADEOFF  INTER-RELATIONSHIPS i  None. 


TASK  SYNOPSIS; 

1.  Task  Requirements. 

The  ideal  testability  FOM  may  be  defined  as  a  numerically  precise  and 
measurable  parameter  that  accurately  evaluates  the  degree  of  testability 
designed  into  the  equipment.  Just  as  MTBF  evaluates  reliability  and  is 
allocable  and  auditable  throughout  the  design  indenture  hierarchy,  so 
too  should  be  the  Testability  Figure  of  Merit.  The  FOM  should  be 
useable  at  any  indenture  level  of  the  system  which  is  subject  to  test. 

1.1  There  is  no  known  and  widely  accepted  FOM  for  Testability.  This  task 
then  is  required  to  screen  suggested  FOMs  and  to  define  the  testability 
FOM(s)  that  will  be  used  to  calculate  the  degree  of  inherent  testability 
designed  into  the  equipment,  to  assess  the  preliminary  design  testability 
by  applying  the  FOM(s),  and  to  take  corrective  action  as  required  before 
PDR. 


2.  Implementation . 

It  appears  that  the  current  state-of-the-art  favors  developing  figures 
which  reflect  observability  of  faults  and/or  controllability  of  test. 

2.1  The  following  measures  of  testability  have  been  suggested  in  the  litera¬ 
ture  without  exact  definitions  and  method  of  calculation.  They  are 
presented  as  concepts  for  further  development  by  the  testability  engineer. 
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Observability  Figures. 


(1) 

FOM 

number 

number 

of  test  points 
of  circuits 

(2) 

FOM 

sum  of 

failure  rates  of  the 

circuits  with  test  points 

sum  of  failure  rates  of  all  circuits 

or 

FOM  - 

sum  of 

failure  rates  of  all 

tested  hardware 

total  failure  rate  of  the  hardware 


(3)  Fault  coverage  =  the  percent  of  faults  detected.  ^ 

(4)  Percent  of  package  or  gates  monitored. 

Combined  Observability/Controllability  Figures. 

(31) 

(1)  fom  «  (%  of  faults  detected)x(%  of  detected  faults  isolated) 

Number  of  SRU's  isolated  directly^!) 

(2)  Shop  Non-Ambiguity  Ratio  -  without  ambiguity 

(LRU  Testing)  Total  number  of  SRU’s  in  the  LRU 

(Task  Reference  Number  V2B  contains  definitions  of  LRD  and  sru.) 

(3)  A  FOM  calculated  from  an  inspection  of  logic  network  parameters 
and  independent  of  the  test  generation  method  1st (23) 

n 

FOM  -  XKjfOTj) 

where:  is  a  scale  factor  and  T^  is  the  observable  factor. 

For  example: 

T^  «  observable  test  access 

T2  -  measure ^of  initialization 

T3  -  measure  of  controllability 
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2.2  The  following  method  inversely  grades  the  degree  of  design  response  to 

testability  requirements  as  a  numerical  value. ^0)  The  method  consists 

of  five  steps. 

STEP  1.  From  the  44  characteristics*  in  Figure  1  create  a  table 
exemplified  in  part  by  Figure  2. 

STEP  2.  For  each  characteristic  fill  in  the  importance  weight 
("IMP  WT")  column  of  the  table  in  Figure  2  using  the 
data  from  Figure  3. 

STEP  3.  For  each  item  fill  in  the  "DESIGN  RESPONSE"  columns  of 
the  table  in  Figure  2  using  the  "NUMERICAL  GRADE"  data 
from  Figure  4. 

STEP  4.  Compute  the  "DIFFICULTY"  column  of  the  table  in  Figure  2 
by  multiplying  the  "IMP  WT"  by  the  "DESIGN  RESPONSE." 

STEP  5.  Plot  the  "DIFFICULTY"  on  Figure  1  for  each  item. 


Figure  1,  when  completed,  provides  indications  of  which  characteristics 
need  most  to  be  improved.  The  grading  system  is  useful  in  identifying 
potential  UUT  testability  problems,  establishing  precise  task  identifi¬ 
cation  and  defining  an  "inverse"  FOM.  (If  the  FOM  were  inverted,  it 
would  better  match  the  conventional  FOM  view  of  the  "bigger  the  better.") 

FOM  =  Sum  of  Difficulty  Factors 

Total  Number  of  Testable  Items 


2.3  A  somewhat  more  detailed  methodology  which  evaluates  the  testability 
merits  of  a  PCB  is  developed  In  Appendix  1.  This  FOM  rating  system 
quantitatively  weighs  the  "difficult  to  test"  and  "easy  to  test"  aspects 
of  a  circuit  design. 


autation  and  Action. 


When  developed,  the  FOM  should  be  calculated,  (if  not  calculated  bottom- 
up)  and  audited  by  the  testability  and  system  engineering  disciplines. 
The  audit  should  identify  design  revisions  that  may  be  required.  Tin 
audit  results  and  agreed  actions  should  be  reviewed  and  approved  or  re¬ 
directed  by  contractor  and  government  program  management.  Continuing 
actions  should  be  assigned  to  the  approximately  responsible  management/ 
engineering  functional  people. 


"Additional  or  alternative  characteristics  may  also  be  used 


I 
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DIFFICULTY 

0.7500 

5.6250 

o 

in 

n 

10 

in 

3.7300 

0.932S 

6.2500 

DESIGN  RESPONSE 

h 

oE 

o  « 

POOR 

3  i 

at- 

Rk 

a 

U.  <M 

* 

* 

GOOD 

1 

pH 

EXCEL 

0 

IMP 

WT 

IT) 

«0 

o 

o 

IT 

CM 

<0 

o 

o 

«n 

CM 

«0 

o 

0.9325 

0.9325 

m 

CM 

IP 

n 

H 

DESCRIPTION  OF 
REQUIREMENTS 

PREFERENCE  FOR  INCLUSION  OF 
TEST  JACKS  FOR  CONNECTION 

OF  EXTERNAL  TEST  EQUIPMENT 
(I.E.,  LOGIC  PROBE,  LOGIC 

PULSER.  ETC!) 

n  ^  r  —  *■ 

■WfSl 

PREFERENCE  FOR 

CPU  BUS  ACCESS. 

PROCESSOR  ISOLATION, 

AND  INITIALIZATION 
PROVISIONS 

PREFERENCE  FOR 

PROVISIONS  RELATING 

TO  POSITIVE  CONTROL 

OF  CLOSED  LOOP 

CIRCUITS 

PREFERENCE  FOR 

PROVISIONS  TO 

CONTROL  INTERNAL 

CLOCKS.  OSCILLATORS, 

ETC. 

REQUIREMENTS  FOR 

QUALITATIVE  DOCUMENTATION 
AVAILABILITY  FOR  ALL 
NECESSARY  ELEMENTS 

OF  TPS  DEVELOPMENT 

COMMON 

TERM 

TEST 

JACKS 

PECULIAR  TEST 
JOINT  INTERFACE 

CPU  FAULT 
ISOLATION 

SERVO 

LOOPS 

STATIC  TEST 
CONDITIONING 

DOCUMENTATION 

ITEM 

number 

N 

(V 

* 

CM 

. 

0t 

CM 

o 

** 

« 

3 
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TERM 

IMPORTANCE 

WEIGHT 

DEFINITION 

EXAMPLE 

CRITICAL 

l.MJ' 

A  CATEGORY  OF 
ITEMS  THAT  ARE 
REQUIRED  TO 
PRODUCE  A  COST 
EFFECTIVE  TEST 

DOCUMENTATION 

VERY 

IMPORTANT 

0.9325 

A  CATEGORY  OF 
ITEMS  THAT  ARE 
REQUIRED  TO 
PRODUCE  AN 
ACCEPTABLE 
LEVEL  OF  TEST 
COMPREHENSION 

mm 

IMPORTANT 

0.625 

A  CATEGORY  OF 
ITEMS  THAT  ARE 
REQUIRED  TO 
ACCOMMODATE 
AUTOMATIC 

TEST 

ATE  INTER¬ 
FACING 

CONSIDERATIONS 

TIME 

RELATED 

0.375 

A  CATEGORY  OF 
ITEMS  THAT 
IMPACT  TEST 
GENERATION 
TIME 

REQUIREMENT 

CONVENIENCE 

RELATED 

0.1875 

A  CATEGORY  OF 
ITEMS  THAT 
RELATE  TO  TEST 
GENERATION 
CONVENIENCES 

PROVISIONED 
EXTERN AT 
EQUIPMENT 
OUTLETS 

Figure  3.  Importance  Weighting  Logic 


NARRATIVE 

GRADE 

NUMERICAL 

GRADE* 

NARRATIVE 

DESCRIPTION 

EXCELLENT 

0 

RESERVED  FOR  THOSE  INSTANCES 
WHERE  DESIGN  RESPONSE  TO 
TESTABILITY  REQUIREMENT  IS 
ADEQUATE  IN  ALL  AREAS  OF 

CONCERN 

GOOO 

2 

RESERVED  FOR  THOSE  INSTANCES 
WHERE  OESIGN  RESPONSE  1$ 
ADEQUATE  IN  AREAS  OF  HIGH 
CONCERN  BUT  NOT  ALL  AREAS 

OF  CONCERN 

FAIR 

4 

RESERVEO  FOR  THOSE  INSTANCES 
WHERE  DESIGN  RESPONSE  IS 
ADEQUATE  IN  SOME  AREAS  OF 

HIGH  CONCERN 

POOR 

9 

RESERVED  FOR  THOSE  INSTANCES 

OF  LITTLE  OR  NO  DESIGN  RESPONSE 
TO  THE  TESTABILITY  REQUIREMENTS 
IN  AREAS  OF  HIGH  CONCERN 

CRITICAL 

L 

16 

RESERVED  FOR  THOSE  INSTANCES 
WHERE  DESIGN  GAVE  NO  CONSIDERA¬ 
TION  TO  THE  TESTABILITY 
REQUIREMENT 

•NUMERICAL  GRADE  •  NC2 


Figure  4.  Design  Response  Grading  Logic 
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APPENDIX  V4B1(22) 

PCB  Testability  Rating  System  Condensation 

The  PCB  rating  system  is  represented  by  Figure  I  (see  next  page) .  Each 
PCB  design  is  rated  on  (i)  four  Basic  Factors,  permitting  a  basic  weighted 
subscore  of  up  to  100  percentage  points  and  (ii)  on  30  "negative"  factors, 
each  of  which  may  subtract  some  finite  number  of  percentage  points  from  the 
Net  Total  Score.  The  net  total  score  assesses  PCB  ease-of -testability  on 
the  following  scale: 

PCB  RATING 


0% 


10% 


30% 


45% 


65% 


80% 


100% 


Very 

- 1 

Medium 

Very 

—L _ 

Hard 

Hard 

Medium 

Easy 

Easy 

Easy 

-Extreme  Cost  Penalties  to  Provide  Test 


To  use  the  rating  system,  true  representation  of  the  actual  PCB  is  required 
in  the  form  of  schematics,  logic  diagrams,  parts  lists,  and  specifications 
and  internal  logic  of  all  PCB  parts. 


The  Basic  Factors 

(1)  B-l  rates  the  accessibility  of  circuit  nodes  oh  the  PCB.  To  apply,  count 
the  number  of  input  and  output  leads,  each  of  which  represents  one 
accessible  node.  Count  also  the  inaccessible  nodes,  e.g. ,  internal  part-to- 
part  leads,  counting  one  for  each  lead.  The  raw  score  for  B-l  is: 

{(number  of  accessible  nodes)  f  [(number  of  accessible  nodes) -(-(number  of 
inaccessible  nodes)]}  X  100%. 

The  count  for  B-l  and  for  subsequent  factors  N-3  and  N-4  can  be  facili¬ 
tated  by  counting  the  parts  connected  to  each  lead,  and  marking  the  appro¬ 
priate  box  of  the  worksheet  shown  as  figure  2.  As  an  example,  assume  a  board 
with  10  external  leads  each  connected  to  5  parts,  6  external  leads  each 
connected  to  15  parts  (10  marks  and  6  marks  respectively  in  the  "5"  and  "15" 
boxes  of  the  "access"  half  of  the  worksheet) ,  5  internal  leads  each  connect¬ 
ing  10  parts  and  10  internal  leads  each  connecting  8  parts  (5  marks  and  10 
marks  in  the  "10"  and  "8"  boxes  in  the  "no  access"  half  of  the  worksheet) . 

The  parts  counts  information  will  be  used  in  N-3  and  N-4. 


The  B-l  raw  score  for  this  example  is  computed  from  the  (10+6)  marks  on 
the  access  half  and  the  (5+10)  marks  on  the  no-access  half  of  the  worksheet  as 

{(10+6)4-  [(10+6)+ (5+10)]}  X100%  =  (16431) X100%  -  52%. 

The  raw  score  is  weighted  for  "actual  rating"  on  Figure  1  as  follows 
on  Page  7.2A-4. 
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FIGURE  1.  PCB  TESTABILITY  EVALUATION  SCORE  SHEET 
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PARTS  (PKGS) 


V4B1-3 


SCORE  % 


ACTUAL 
RATING  % 


(2)  B-2  rates  six  features  of  the  PCS  documentation  with  maximum  weighted 

(actual  rating)  percentage  points  as  follows: 

% 

FEATURE  (MAX) 

e  Logic  diagrams  or  schematics  (of  all  detailed  parts) 

provided  either  on  overall  print  or  as  individual  4 

part  specifications 

e  Detailed  performance  specification  with  signal  I/O 

tolerances  provided  8 

e  Truth  table  for  each  digital  le  circuit  type  shown 

on  schematic  or  on  detailed  part  drawing  provided  3 

e  Functional  designations  should  be  shown  next  to  each 

pin  number  of  all  logic  packages  on  the  schematic  5 

e  Power  circuits  shown  in  a  single  location  on  the 

schematic  and  with  voltages  labeled  3 

e  Schematic  shows  reference  to  corresponding  assembly 

print  and  part  number  of  next  higher  assembly  2 


(3)  B-3  rates  the  sequential  circuit  content  of  the  PCB.  Using  the 

schematics  and  the  PCB  parts  list,  add  up  the  total  number  of 
sequential  IC  packages,  and  divide  by  the  total  of  all  IC  packages.  Count 
functional  groups  of  discrete  parts  as  equivalent  to  one  1C.  Convert  the 
percent  of  sequential  circuits  ("Score")  using  scale  factors  below  to  get 
the  actual  rating.  Each  integrated  circuit  package  on  the  schematic  is 
counted  as  a  single  sequential  or  combinational  circuit  regardless  of  its 
individual  complexity. 


1 91-100 
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0-10  | 

30 

27 

24 

21 

18 

15 

12 

8 

n 

n 

SEQUENTIAL  CIRCUITS  %  SCORE 
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(4) 


B-4  evaluates  overall  PCS  complexity.  This  count  is  made  for 
sequential  circuits  only,  neglecting  combinational  ZCs.  Each  circuit 
configuration  type  is  scored  according  to  the  counts  in  the  table  below. 


Logic  Type  Score  Counts 

e  Flip  Flop  7  e  4-Bit  Shift  Register  35 

e  Latch  7  e  Memory  Chip  (n  inputs)  2n 

e  VLSI  Chip  or  micro¬ 
processor  1000 

e  For  con?) lex  ic  circuits  with  sequential  sections,  count  points 
for  each  internal  combinational  gate  *  number  of  input  leads 
plus  one  and  each  inverter  ■  3.  Then  sum  the  counts  for  internal 
gates  with  the  counts  for  the  flip/flop,  latch,  chip  and  shift 
register  logic  types. 

The  raw  score  is  weighted  to  actual  rating  as  follows: 


COMPLEXITY 

COUNT 

ACTUAL 
RATING  « 


Compilation  of  Negative  Factors 

Each  of  30  possible  negative  factors  is  scored  in  a  straightforward  way  for 
entry  to  Figure  3. 

N-l  Classify  each  monostable  into  one  of  three  categories: 


Category  for  Scoring 

Actual 
Rating  % 

(a) 

Is  tested  by  analog  techniques  not  requiring 
digital  ATG  processing 

(-  1)  per 
instance 

(b) 

Accessible  monostable  output  driving  sequential 
circuits 

(-  2)  per 
instance 

(c) 

Inaccessible  monostable  output  driving 
sequential  circuits 

(-  5)  per 
instance 

N-2  Multiply  the  number  of  XC  packages  by  the  number  of  internal 

sequential  stages.  The  count  of  stages  starts  from  each  direct 
input  and  continues  until  the  final  stage  of  the  counter  is  reached,  or 
until  a  point  is  reached  where  another  input  can  be  injected. 
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Up  to  300 

301-500 

501-800 

801-1200 

1201-1800 

1800  + 

20 

16 

12 

8 

4 

0 

N-3  From  the  bottom  half  of  the  worksheet  (Figure  2)  count  instances 

where  more  them  3  different  function  blocks  (circuit  packages)  are 
connected  to  the  same  (internal)  wiring  junction  (node) . 


Inaccessible  Nodes 

Actual 
Rating  % 

(a) 

4 

(-0.1)  per 
instance 

(b) 

5 

(-0.2)  per 
instance 

(c) 

6 

(-0.5)  per 
instance 

(d) 

7 

(-1.0)  per 
instance 

(e) 

8 

(-1.3)  per 
instance 

(f) 

9 

(-1.7)  per 
instance 

(g) 

10  and  higher 

(-2.0)  per 
instance 

N-4  Repeat  the  N-3  procedure  for  all  accessible  nodes  with  5  or  more 
packages  tied  together. 

Actual 


Accessible  Nodes 

Rating  % 

(a) 

5 

(-0.1)  per 
instance 

(b) 

6 

(-0.2)  per 
instance 

(c) 

7 

(0.5)  per 
instance 

(d) 

8 

(-0.6)  per 
Instance 

V4B1-6 


N-4  continued 


(e)  9 

(f)  10  and  higher 


(-0.8)  per 
instance 
(-1.0)  per 
instance 


H-5  If  two  or  more  supply  voltages  require  a  turn -on  and/or  turn-off 
sequence,  assess  a  -10%  penalty. 


N-6  Any  type  of  memory  permanently  wired  to  the  PCB  with  all  I/O  leads 
accessible  is  penalized. 


Actual 

Memory  Size  (BITS) 

Rating  « 

(a) 

100K  and  over 

(-10)  per 
instance 

(b) 

32K  to  99K 

(-6)  per 
instance 

(c) 

8K  to  3 IK 

(-4)  per 
instance 

(d) 

IK  to  7K 

(-2)  per 
instance 

N-7  Any  memory  permanently  wired  to  the  PCB  with  one  or  more  of  its 
leads  not  connected  to  I/O  pins  is  penalized i 


Memory  Size  (BITS) 


Actual 
Rating  % 


(a)  Under  IK  (  -5)  per 

instance 

(b)  -  IK  (-10)  per 

instance 


N-8  A  -1%  penalty  is  imposed  per  instance  of  a  part  mounted  in  a 

socket  or  the  equivalent  and  requiring  extraction  prior  to  test 
access. 
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N-9  Non-removable  microprocessors,  VLSI  chips  or  other  complex  parts 
are  penalized: 


Scoring  Factors 

Actual 
Hating  % 

(a) 

All  leads  accessible  to  I/O  pins 

(  -3)  per 
instance 

(b) 

One  or  more  leads  not  accessible  to 

I/O  pins 

(-10)  per 
instance 

N-10  Check  each  sequential  circuit  to  see  if  it  can  be  initialized  in 

two  ways;  using  the  direct  set/reset  inputs,  and  using  signal  input 
of  less  than  16  patterns  with  a  clock  line.  Penalize  each  case  where 
initialization  cannot  be  accomplished  in  two  ways: 


Actual 


Scoring  Factors 

Rating  % 

(a) 

Direct  set  and  <16  pattern  reset 

No  penalty 

(b) 

Direct  set  but  no  pattern  reset 

(-0.05)  per 
instance 

(c) 

No  direct  set  but  <16  pattern  reset 

(-0.1)  per 
instance 

(d) 

No  direct  set  and  >16  pattern  reset 

(-2)  per 
instance 

N-ll  If  external  loading  is  required  by  components  which  must  be  added 
to  the  Interface  Device  to  perform  test  (e.g. ,  pullup  resistors) , 
penalize  as  follows: 


Actual 

Scoring  Factors 

Rating  % 

(a) 

10  resistive  loads 

(-2) 

(b) 

50  and  over  resistive  load 

(-3) 

(c) 

>5  Reactive  Loads 

(-5) 

N-12  Diversity  of  IC  type  numbers  is  penalized: 

Actual 


Scoring  Factors 

Rating  % 

(a) 

7  types 

No  penalty 

(b) 

10  types 

(-1) 

(c) 

>10  types 

(-1)  for  each 
additional  3 
types 
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{ 
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N-13  Start  with  any  sequential  circuit  (count  of  1)  and  count  each 
sequential  stage  directly  connected  to  one  of  its  inputs  or  to 
one  of  its  outputs.  If  an  output  lead  from  an  otherwise  unconnected 
sequential  circuit  is  connected  to  the  clock  input  of  a  sequential  circuit 
in  the  above  cluster,  it  should  also  be  counted.  Expand  the  count  in  all 
directions  until  all  signal  leads  from  all  circuits  in  the  cluster  reach 
combinational  circuits  or  a  PCB  input/output  lead.  Continue  this  process 
until  all  sequential  circuits  have  been  counted. 


Assess  penalties  for  each  cluster  of  three  or  more 
circuits  as  shown:  (Do  not  count  2n  buried  counters  under 


sequential 
this  step.) 


Actual 


Scoring  Factors 

Rating  % 

(a)  Cluster  of  3  or  4  sequential  circuits 

(b)  Cluster  *5 

(-0.1)  per 
instance 
-0.1(l+(N-5) ) 
per  instance 

N-14 

Input-Output  pins  distinguished  on  schematic  makes 
signal  paths  easier. 

tracing  of 

Scoring  Factor 

Actual 

Rating  % 

(a)  Direction  arrows  not  different  for  input 
pins  versus  output  pins 

(-3) 

N-15 

Harm-up  time  required  :o  stabilise  card  should  not 

exceed  3  minutes 

Scoring  Factor 

Actual 

Rating  % 

(a)  Over  3  minutes 

(-3) 

N-16 

Test  equipment  tolerance  (if  test  equipment  characteristics  are 
known)  is  rated  as  follows: 

Scoring  Factors 

Actual 

Rating  % 

(a)  Measurement  capability  at  least  10  times 
more  accurate  than  PCB  requirement 

No  penalty 
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N-16  continued 


(b)  Measurement  capability  3  times  more 
accurate  than  PCB  requirement 

(c)  Measurement  capability  less  than  3  times 
more  accurate  them  PCB  requirement 


N-17  High  power  requirements  are  penalized: 

Scoring  Factors 

(a)  More  than  5  amps  of  current  required 

(b)  High  voltage  >300Vpp 

(c)  Multiple  parallel  pins  for  high  current 

N-18  If  frequency  is  critical  to  measurement: 


Scoring  Factors 

(a) 

Requires  co-ax  in 

I~t*.rface  Device 

(b) 

Over  10  MHz 

(c) 

Over  4  MHz 

(d) 

Over  1  MHz 

Clock  lines  complexity 

is  penalized: 

Scoring  Factors 

(a)  One,  externally  controlled 

(b)  Multiphase,  externally  controlled 

(c)  Single  clock,  monitor  only 

(d)  Multiple  clocks,  monitor  only 

(e)  Inaccessible  free-running  clock 


(-2)  per 
instance 
(-5)  per 
instance 


Actual 
Rating  % 

(-5)  per 
instance 
(-2)  per 
instance 
(-1)  per 
instance 


Actual 
Rating  I 

(-5) 

(-3) 

(-2) 

(-1) 


Actual 
Rating  % 

(-1) 

(-2) 

(-3) 

(-5) 

(-20) 
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N-20 


Test  equipment  other  than  that  contained  in  the  automatic  test 
equipment: 


Actual 

Scoring  Factors 

Rating  % 

(a) 

2  power  supplies  or  more 

(-2) 

(b) 

Oscilloscope 

(-2) 

(c) 

Function  Generator 

(-4) 

N-21  Special  chambers  or  areas  required  to  perform  test: 


Scoring  Factors 


Actual 
Rating  % 


(a)  Forced  air,  ambient  or  chilled  (-2) 

(b)  Heat,  altitude,  EMI  (chamber)  (-10) 


N-22  Adjustments  required,  as  for  trimpots  and  variable  capacitors, 
are  penalized: 

Actual 

Scoring  Factors  Rating  % 

(a)  per  instance  (-2) 

(b)  per  interactive  adjustment  (-4) 


N-23  Interpretation  by  the  test  operator  is  required  where  complex  or 
non-periodic  waveshapes  are  used  is  penalized: 


ACtUa.  1 

Scoring  Factors 

Rating  a 

(a) 

2  coincident  unusual  waveforms 

(-5  per 
instance) 

(b) 

1  unusual  waveform 

(-2  per 
instance) 
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N-24  Logic  which  in  parallel  prevents  fault  isolation  and/or  detection 
of  individual  logic  failures,  unless  built- in-test  permits  fault 
isolation  of  redundant  elements,  is  penalized: 


Actual 


Scoring  Factors 

Rating  % 

(a) 

2  parallel  logic  functions  -  inseparable 

(-2)  per 

instance 

(b) 

3  and  over  parallel  logic  functions  - 

(-3)  per 

inseparable 

instance 

Excessive  logic  voltages  are  penalized: 

Actual 

Scoring  Factors 

Rating  % 

(a) 

4 

No  penalty 

(b) 

>4 

(-1  per 

additional 
logic  voltage) 


N-26  Excessive  numbers  of  separate  power  supplies  which  must  be  supplied 
by  the  test  station  are  penalized: 


Scoring  Factors 


Actual 
Rating  % 


(a)  3 

(b)  >3 


No  penalty 
(-1  each 
additional 
supply) 


N-27  The  aim  of  this  factor  is  to  guarantee  that  the  schematic/logic 
diagrams  do  not  impose  hardship  on  the  test  design  engineer. 


Scoring  Factors 


Actual 
Rating  % 


(a)  Schematic  on  single  page 

(b)  If  schematic  on  multiple  pages  with 
connecting  leads  between  pages  -  then  all 
interpage  connectives  are  numbered  showing 
other  page  numbers  and  zones 

(c)  If  neither  (a)  nor  (b)  condition  is  met 


No  penalty 
No  penalty 


(-20) 
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N-28 


I/O  pins  located  in  the  center  of  prints  cause  extra  work  for 
test  designer. 


Actual 

Scoring  Factor  Rating  % 

(a)  All  I/O  pins  not  brought  to  edges  of  schematic  (-5) 
diagram  or  to  a  common  dotted  line 


N-29  Dual  I/O  pin  designation 

Actual 

Scoring  Factor  Rating  % 

(a)  If  dual  designation  of  aui  I/O  pin  is  in  (-3)  per 

different  areas  of  print  with  no  cross-  instance 

reference 


N-30  Only  a  single  symbol  should  be  used  to  describe  a  specific  hardware 
part.  Multiple  symbols  for  identical  parts  make  it  difficult  to 
check  ATG  bit  propagation  and  to  design  key  manual  patterns  to  supplement 
tests. 

Actual 

Scoring  Factor  Rating  % 

(a)  IC  Logic  Symbols  used  are  not  identical  to  (-5) 

detail  part  drawing  symbols 


Totaling  and  Evaluation 

After  evaluating  all  factors,  the  Basic  and  Negative  scores  can  be  combined 
for  reference  to  the  PCB  Testability  Rating  shown  in  the  opening  paragraph 
above. 


i 


t 
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PHASE: 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V4C 


VALIDATION 


FUNCTION:  Inherent  Testability  Analysis  of  Preliminary  Design 

task  TITLE:  Analyze  hardware/software  BIT  features 

TASK  OBJECTIVE:  Document  the  tradeoffs  made  in  selecting 

„  hardware  BIT  features  using  the  tradeoff  analysis 
data  base  for  participation  in  a  design  review  to  determine  the  inclusion 
of  suitable  built-in-test  features. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE _  INPUT  TO  T _  OUTPUT  FROM  T _ 


e  Design 
Engineering 

e  Application 
Software 


e  HW  BIT  Features,  Schematics 
Component  Listings,  Chassis 
Diagram 

e  SW  BIT  Features,  Program 
Lists 


e  T  Analysis  Problem  Areas 
and  Solutions 

e  T  Analysis  Problem  Areas 
and  Solutions 


e  Life  Cycle 
Costing 


e  HW/SW  Tradeoffs  Cost 
Guidance 


e  T  Analysis  Costing 
Data 


COST  TRADEOFF  INTER-RELATIONSHIPS :  Cost  relationships  enter  this  task  only 

indirectly,  since  the  major  cost  tradeoffs  occur  in  conjunction  with  task 
functions  V2  and  V3  (system  design  and  BIT  tradeoffs) . 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  following  BIT  tradeoffs  will  be  documented. 

a.  Placement  of  BIT  failure  indicators 

b.  Use  of  standard  components  to  implement  BIT 

c.  Use  of  modular,  flexible  BIT  designs 
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d.  Use  of  active  stimulus  injection  for  BIT 

e.  Use  of  circuitry  to  check  BIT  circuitry 

f.  Use  of  circuitry  to  override  BIT  failure  indications 

g.  Use  of  on-line  (nondisruptive)  testing  and  off-line  (disruptive) 
testing. 

h.  Use  of  hardware,  software  and  firmware  BIT  \ 


1.2  Assess  the  inclusion  of  test  requirements  in  the  sizing  of  the  memory. 

\ 

a.  word  allocation.  Insure  that  sufficient  words  are  allocated. 

(1)  In  control  memory  for  the  storage  of  microdiagnostics  and 
initialization  routines. 

(2)  In  main  memory  for  the  storage  of  error  processing  routines. 

(3)  In  secondary  memory  for  the  storage  of  diagnostic  routines. 

b.  Byte  allocation.  Insure  that  a  sufficient  number  of  bytes  are 

assigned  to  each  word. 

(1)  In  control  memory  to  achieve  required  controllability  of 
hardware  components. 

(2)  In  main  and  secondary  memory  to  provide  for  error  detection/ 
error  correction  techniques,  as  required. 


c.  Protection  allocation.  Insure  that  a  sufficient  number  of  memory 
words  jure  assigned  to  non-alterable  memory  resources  (e.g. ,  Read 
Only  Memory,  protected  memory  areas)  to  insure  the  integrity  of 
critical  test  routines  and  data  and  that  sufficient  hardware  and 
software  redundancy  exists  to  confidently  load  critical  software 
segments. 


2.  Implementation . 

2.1  In  all  interfaces,  the  testability  engineer  should  be  prompt,  active 

and  assertive.  Testability  is  most  effectively  instilled  into  a  system 
if  design  guidelines  are  provided  when  design  commences.  It  is  in¬ 
efficient  for  the  testability  (or  any  other  -ility)  engineer  to  take  the 
role  of  a  post-design  critic  and  difficult  as  well  as  uneconomical  for 
designers  to  incorporate  changes  to  documented  designs  when  those  changes 
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reflect  design  criteria  which  could  have  and  should  have  been  disclosed 
during  or  ahead  of  design  formulation.  Timeliness  is  therefore  a 
principal' responsibility  for  the  testability  engineer.  BIT  hardware/ 
software  design  guidance  should  be  provided  by  the  Testability  engineer 
for  the  designers'  'ise  at  the  outset  of  the  design  task. 


2.2  The  analysis  involves  locating  testability  elements  on  chassis  drawings, 
schematics,  functional  flow  diagrams,  software  logical  flow  diagrams  and 
listings  and  the  standard  components  list.  The  next  step  is  to  compare 
the  baseline  configuration  with  other  possible  mechanizations.  For 
example: 

a.  Is  the  BIT  result  indicator  located  at  a  convenient  place  for 
sighting  by  the  maintenance  technician  during  maintenance? 

b.  Is  the  BIT  circuitry  mechanization  fail  safe? 

c.  Is  memory  sizirg  adequate  for  near-term  increased  requirements? 


2.3  During  the  initial  design  stage,  the  proposed  BIT  features  should  be 

analyzed  and  the  resulting  reconnendations  applied  to  the  design.  This 
iterative  process  should  continue  until  the  design  incorporates  the 
required  BIT  features.  Completion  of  this  task  occurs  upon  signoff  of 
the  design  (at  base  line  design) . 


3.  Completion  Criteria. 

The  task  is  considered  complete  upon  government  approval  of  the  BIT 
Features  Analysis  in  conjunction  with  the  Preliminary  Design  Review. 


PHASE: 


TESTABILITY  TASX  COMPENDIUM 
Task  Reference  Number  V4D 

VALIDATION 


FUNCTION:  Inherent  Testability  Analysis  of  Preliminary  Design 

TASK  TITLE:  Conduct  testability  analysis  of  the  potential  OUTs  in  the 

preliminary  design 

TASK  OBJECTIVE:  Determine  the  extent  to  which  testability  design  require¬ 

ments  and  guidelines  provided  to  design  activities  in 
Functions  V2  and  V3  have  been  incorporated  in  design,  and  provide  guidance  for 
subsequent  detail  design. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE _ INPOT  TO  T  OUTPUT  FROM  ¥ 


e  System 
Engineering 


e  Partitioning,  System  Level  e  T  Analysis  Results 
BIT/ETE,  System  Functional  (recommendations) 

Diagram 


e  Design 
Engineering 


e  OUT  Test  Points,  HW  BIT, 
Inherent  T  Features, 
Schematics,  Logic  Diagrams 


•  T  Analysis  Results 
( recommendations) 


e  Application 
Software 


e  SW  BIT,  Test  Program  Flow 
Charts  and  Program  Lists 


e  T  Analysis  Results 
(recommendations) 


e  Reliability 
Engineering 


e  FMEA,  System/Nodule 
Failure  Rates 


e  T  Analysis  Results 
(recoonendations ) 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Cost  tradeoffs  are  involved  only 

indirectly  in  this  task,  in  that  the  task  is  a  followup  to  those  tasks  which 
gave  consideration  to  cost  tradeoffs. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  A  formal  analysis  and  documentation  of  the  inherent  testability  of  the 
preliminary  design  is  required  as  a  followup  to  the  design  interfacing 
conducted  in  Function  V2,  Incorporation  of  Testability  into  System  Design 
and  Function  V3,  Performance  of  Test  Method  Tradeoffs.  The  characteristics 
to  be  given  primary  consideration  are  Observability,  Controllability  and 
Testability  Figures-of-Merit. 
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The  analysis  nay  be  based  on  a  hierarchical  structured  format  represent¬ 
ing  the  prime  system  through  a  configuration  listing  with  an  associated 
set  of  qualitative  and  quantitative  testability  parameters.  The  format 
thus  describes  the  Testability  configuration  of  the  validation  phase 
prime  system  design  and  can  be  used  in  full  scale  development  to  improve 
the  testability  of  the  detail  design  by  comparing  that  design  with  the 
preliminary  design. 

A  format  displaying  allocated,  predicted  and  achieved  testability 
characteristics  should  be  constructed  for  each  testable  configuration 
item  (i.e.,  potential  UUT)  at  each  successive  level  of  the  prime  system 
hierarchical  indenture.  The  formats  are  to  be  used  extensively  by  the 
contractor  in  performing  design  work  and  are  convenient  tools  for  use  by 
the  government  in  review.  The  results  of  the  analysis  should  be  incor¬ 
porated  into  the  Testability  Analysis  Report. 


2.  Implementation . 

Functions  V2  and  V3  provide  baseline  data  for  use  in  this  task.  Given  that 
the  Tasks  of  Function  V2  and  V3  have  been  properly  performed ,  the  test¬ 
ability  analysis  may  well  commence  on  a  quantitative  level.  If  there  has 
been  no  prior  interface,  analysis  of  the  design  for  inherent  T  should 
begin  on  a  qualitative  level,  then  as  actions  are  taken  with  system/ 
design  engineering  to  correct  identifiable  system  design  shortcomings, 
the  analysis  can  continue  to  quantitative  levels. 


2.1  Qualitative  Analysis  Checklist. 

If  the  analysis  is  co  be  initiated  on  a  qualitative  basis,  the  qualita¬ 
tive  analysis  should  begin  with  a  system  testability  checklist  which 
would  include  as  a  minimum  determining  the  extent  of  the  following  design 
features: 

a.  Physical  partitioning 

b.  Functional  partitioning 

c.  Regular  structured  designs 

d.  Initialization 

e.  Observability 

f.  Controllability 

g.  System  compatibility  with  ETE  resources 

h.  Suitable  BIT  features  for  planned  maintenance  concepts 

i.  Suitable  controls  and  displays  to  provide  required  human  interface 
for  tests  and  maintenance  actions. 

2.2  General  Notes  for  Detailed  Quantitative  Analysis. 

a.  The  detailed  analysis  should  include  narked  functional  schematics 
which  indicate  the  control  that  the  test  system  has  over  the  prime 
equipment  items  and  the  access  that  the  test  system  has  to  the 
internal  state  of  the  prime  items. 


b. 


An  analysis  of  failure  modes,  their  effects  on  each  item,  and  the 
ability  fcr  BIT,  or  ETE  to  fault  isolate  each  failure  should  also 
be  included. 


2.3  Testability  Analysis  Modeling/1* 

The  following  excerpt  is  adapted  from  reference  (1)  and  provides  somewhat 
theoretical  guidance  for  analyzing  testability  in  a  bottom-up  mode  which 
is  also  susceptible  to  computer  assist.  The  details  of  the  computer 
assist  are  not  defined. 

"Computer-assisted  analysis  modeling  is  based  on  the  theory  that  the 
design  of  testable  modules  is  straightforward  when  their  conponents  are 
themselves  testable;  the  design  of  testable  assemblies  is  straightforward 
when  their  modules  are  each  testable;  and  so  on  up  to  the  system  level. 
The  design  process  at  each  level  is  reduced  to  chat  of  providing  patterns 
to  the  input  of  each  element  and  prop.-  jating  the  known  failure  effects 
through  the  assumed  fault-free  components  to  observable  outputs. 

"The  test  development/test  measurement  process  starts  with  analysis  of 
the  several  basic  hardware  entities  for  which  fault  simulation  is  per¬ 
formed.  The  stimulus/response  data  and  testable  design  features  required 
to  meet  specifications  are  ascertained  through  an  iterative  process.  This 
lowest  level  of  testability  design  and  analysis  may  be  referred  to  as  the 
basic  Testability  Building  Block  (TBB) .  A  TBB  may  or  may  not  correspond 
to  a  physical  or  functional  partition  of  the  system. 

"Higher  levels  in  the  testability  design  and  analysis  hierarchy  would  be 
defined  in  terms  of  the  lower  level  blocks  and  the  stimulus/response 
requirements  for  each  lower  level  block.  Any  such  definition  may  be 
referred  to  as  a  Testability  Analysis  Model  (TAM) .  A  TAM  may  correspond 
to  a  module,  subassembly,  assembly,  or  to  any  physical  or  functional 
grouping.  TAMs  should  be  chosen  so  as  to  contain  approximately  the  same 
degree  of  complexity,  independent  of  hierarchical  level.  Higher  level 
TAMs  contain  more  complex  blocks  but  that  complexity  is  transparent  with 
respect  to  testability  analysis.  The  TAM  is  also  potentially  useful  as 
a  top-down  design  tool  for  testable  systems. 

"Each  TAM,  automated  or  not,  should  permit  the  following  analysis; 

Define  unambiguously  the  test  hierarchy  for  this  level. 

Identify  the  response  observation  points  (for  BIT  and  ETE)  for 
this  level. 

Evaluate  the  effectiveness  of  error  monitoring  circuits. 

Identify  any  special  test  modes  and  stimulus  injection  points 
for  this  level. 

Verify  that  data  paths  and  control  paths  exist  to  stimulate 
lower  level  blocks  as  required,  and 
Verify  that  data  paths  and  control  paths  exist  to  propagate  the 
response  of  lower  level  blocks  to  observation  points. 


"The  process  required  to  support  such  a  hierarchy  of  test  levels  and  test 
analysis  is  neither  a  well-defined  nor  a  well-developed  process.  Although 
the  active  simulation  of  faults  is  not  performed  beyond  the  lowest  level 
(the  TBB) ,  the  generation  of  required  patterns,  the  calculation  of  re¬ 
sponses,  and  the  compilation  of  comprehensiveness  data  for  the  higher 
levels  is  not  a  trivial  task.  The  process  undoubtedly  requires  automa¬ 
tion  for  analyzing  systems  of  moderate  size  or  larger.  The  process 
required  would  be  closely  related  to  the  process  of  deductive  simulation 
developed  for  digital  circuits  but  may  well  be  adaptable  to  non-digital 
circuits  as  well.,." 


2.4  Analysis  and  Documentation  Guidelines  Notes 

The  following  general  guideline  information  is  adapted  from  reference  (1) . 

a.  Figures  of  merit.  Figures  of  merit  for  inherent  testability  may 
be  developed  as  feasible  and  applied  to  the  potential  UUT  represented 
by  each  Testability  Analysis  Model.  (Refer  to  Validation  phase 
Task  Conpendium  V4B  for  Figures  of  Merit.) 

b.  Analysis  of  built-in-test  features.  The  preliminary  design  should 
include  suitable  built-in-test  features.  The  following  BIT  trade¬ 
offs  should  be  documented: 

(1)  Placement  of  BIT  failure  indicators 

(2)  Use  of  standard  components  to  implement  BIT 

(3)  Use  of  modular,  flexible  bit  designs 

(4)  Use  of  active  stimulus  injection  for  BIT 

(5)  Use  of  circuitry  to  check  BIT  circuitry 

(6)  Use  of  circuitry  to  override  BIT  failure  indications 

(7)  Use  of  on-line  (nondisruptive)  testing  and  off-line 
(disruptive)  testing 

(8)  Use  of  hardware,  software  and  firmware  BIT 

c.  Documentation  of  testability  characteristics. 

The  qualitative  description  of  testability  features  incorporated 
during  preliminary  design  should  be  included  in  the  preliminary 
Testability  Analysis  Report.  This  report  serves  as  a  single  source 
of  information  on  testability  requirements,  tradeoffs,  and  functional 
testability  design  for  review  at  PDR. 

(See  Validation  Phase  Task  Reference  Number  V5A) 


3.  Completion  Criteria. 

The  task  is  complete  upon  government  acceptance  of  the  documented  test¬ 
ability  analysis  results.' 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V5A 


PHASE:  VALIDATION 

FUNCTION:  Testability  Analysis  Report  Preparation 

TASK  TITLE:  Prepare  Testability  Analysis  Report 

/ 

TASK  OBJECTIVE:  Prepare  a  single  document  which  represents  the  status 

and  posture  of  the  design's  inherent  testability  as  of 
the  Preliminary  Design  Review. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


e  System 
Engineering 

e  Design 
Engineering 


e  Partitioning,  BIT  Tradeoffs, 
System  Level  BIT/ETE 


e  Testability  Development 
Specification  Paragraphs 


e  Maintainability 
Engineering 


e  Application 
Software 

e  Reliability 
Engineering 


e  Partitioning,  Initialization, 
Controllability,  UUT  Test 
points,  BIT  for  Fault  Detec¬ 
tion  &  Isolation,  Preventive 
Maintenance,  BIT  Features  and 
parts  selection  for  Testability 

e  Maintainability  Program  Plan 
and  integrated  logistics 
support  plan 


e  Testability  Development 
Specification  Paragraphs 


e  Direct  test  sequences 


e  FMEA,  Failure  Mode 
Frequency 


e  cpci  Testability 
Paragraphs 


e  Life  Cycle 
Costing 


e  Cost  Data 


e  Program  Manager 


e  Testability  Analysis 
Report 


COST  TRADEOFF  INTER- RELATIONSHIPS : 


Hone  directly  applicable  to  report 
preparation. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

(5) 

1.1  The  baseline  sections  of  the  report  include:' 

a.  A  description  of  the  partitioning  used  to  enhance  testability. 

b.  A  brief  functional  description  of  each  applicable  hardware  item. 

c.  An  analysis  of  potential  failure  modes  and  effects  for  each  item. 

Data  on  failure  rates  and  confidence  levels  may  be  referenced  from 
the  Reliability  Program,  as  applicable. 

d.  A  summary  of  the  overall  maintenance  concept  taken  from  the  Mainta-- 
ability  Program  and  Integrated  Logistics  Support  Plan  and  a  do sera 
tion  of  the  overall  test  strategy  to  implement  the  maintenance  cct  t 
including  coordination  between  BIT  and  ETE . 

e.  A  description  of  the  test  strategy  to  oe  used  for  each  applicable 
item,  as  determined  by  the  overall  test  strategy. 

f.  A  functional  description  of  built-in  test  features  including  hardwv  , 
and  software  BIT. 

g.  A  functional  description  of  testability  features,  including  control¬ 
lability  and  observability  considerations,  based  upon  the  preliminary 
Testability  Analysis  Model  for  each  item. 

h.  A  functional  description  of  testability  measurement  techniques, 
including  computer-aided  analysis  tools,  to  be  used  during  detail 
design. 

i.  A  description  of  the  methodology  used  for  allocating  quantitative 
system/equipment  testability  requirements  to  the  detail  design  of 
lower  level  items. 


2.  anglementation . 

2.1  The  testability  analysis  report  serves  as  a  single  source  of  testability 
information  and  is  directly  usable  by  specialty  engineering  personnel 
within  the  originating  company,  support  contractors  and  the  government. 

2.2  The  following  excerpt  from  the  proposed  Data  Item  Description  (DID)  for 
the  Testability  Analysis  Report,  contains  the  report  Preparation 
Instructions. These  are  additional  to  the  baseline  sections  in 
paragraph  1.1  above. 
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2.2.1  The  qualitative  sections  of  the  Testability  Analysis  Report  shall 
include  those  items  listed  in  paragraph  1.1  above. 


2.2.2  The  quantitative  sections  of  the  Testability  Analysis  Report  shall 
-include : 

a.  For  each  applicable  hardware  item,  a  description  of  the 
Testability  Analysis  Model  including: 

(1)  A  definition  of  the  failure  population  in  accordance 
with  the  specification. 

(2)  Identification  of  the  test  stimulus,  including  built-in 
test  stimulus  and  external  stimulus. 

(3)  A  determination  of  the  percentage  of  failures  in  the 
failure  population  which  are  detected  by  the  test 
stimulus . 

(4)  A  determination  of  the  level  of  fault  isolation  achievable 
with  the  test  stimulus. 

(5)  The  justification  for  classes  of  failures  remaining 
undetected  or  which  are  poorly  isolated. 

b.  For  the  overall  system: 

(1)  A  description  of  the  integration  of  the  items  and  their 
test  stimulus/response  at  the  system  level. 

(2)  A  determination  of  the  overall  system  fault  coverage 
and  level  of  fault  isolation  based  upon  an  appropriate 
combination  of  these  characteristics  for  each  item. 

(3)  An  estimate  of  developmental  and  recurring  costs 
associated  with  design  for  testability,  including  weight, 
volume,  and  reliability  penalties,  and  increased  computer 
memory  requirements. 

(4)  An  estimate  of  developmental,  production,  and  support 
savings  associated  with  design  for  testability,  including 
reduced  checkout  time,  training,  spares,  and  ETE  costs. 


2.3  The  following  excerpt  from  the  proposed  DID  for  the  Testability 

Analysis  Support  Data  contains  the  data  Preparation  Instructions. d) 


2.3.1  The  Testability  Analysis  report  data  shall  reflect  each  item's 
current  configuration  and  include: 

a.  Testability  Analysis  Model  for  the  item 

(1)  Component  characteristics 

(2)  Component  failure  models 

(3)  Component  interconnection  data 

(4)  Test  control  nodes 

(5)  Test  observation  nodes 

(6)  Interrelationship  of  models 

b.  Testing  Data  for  the  item 

(1)  Test  stimulus 

(2)  Predicted  good  response 

(3)  Predicted  fault  responses 

(4)  Test  stimulus  restrictions 

(5)  Test  response  tolerances 
(6}  Fault  coverage  data 

(7)  Fault  isolation  data 


3.  Completion  Criteria. 

3.1  The  Testability  Analysis  Report  will  be  contractually  required  by 
CDFL/DID.  The  degree  of  acceptability  can  be  assessed  by: 

a.  Degree  of  adherence  to  the  DID  requirements. 

b.  Using  the  topic  checklist  in  the  design  requirements  as 
a  gauge  of  completeness . 

c.  Having  the  writing  services  quality  assurance  function  review 

the  completed  plan  for  editing  principles,  clarity  and  conciseness. 

The  task  is  completed  upon  government  acceptance  of  the  report. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V6A 


PHASE: 


VALIDATION 


FUNCTION: 

TASK  TITLE: 


TASK  OBJECTIVE: 


Development  Specifications 

Prepare  Testability  inputs  to  Configuration  Item  (Cl) 
development  specifications. 


Assure  that  requirements  for  testability  features  are 
properly  included  in  the  Cl  specifications. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE  INPUT  TO  T  OUTPUT  FROM  T 


e  System  e  System  T  Requirements 

Engineering 

e  Design  Engineering  e  T  Requirements  for  each 

Cl 


e  Support  Equipment 


e  T  Requirements  for  ETE 
Developed 


COST  TRADEOFF  INTER-RELATIONSHIPS:  There  are  no  significant  cost  tradeoffs 

involved  in  the  task  of  specification  preparation.  However,  the  contents  of 
the  specifications  must  be  consistent  with  the  achievement  of  cost  effective¬ 
ness. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

It  is  necessary  that  the  testability  requirements  for  failure  analysis, 

BIT  resources,  compatibility  with  ETE,  test  point  access,  observability, 
controllability,  and  initialization  (where  applicable)  be  included  in  the 
development  specifications  for  each  Configuration  Itsm  (Cl) . 

1.1  Equipment  T  Design  Requirements. 

Refer  to  the  tasks  of  functions  V2,  V3,  and  V4  for  equipment  design  require¬ 
ments  and  attributes.  The  testability  requirements  developed  in  these 
tasks  as  applicable  to  the  Cl  should  be  included  within  the  Cl  develop¬ 
ment  specifications. 
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1.2  Tolerance  Cone 


In  conjunction  with  the  specifications  development,  it  is  appropriate  to 
establish  a  tolerance  cone,  as  shown  in  figure  1,  to  preclude  test  incon¬ 
sistencies  between  any  of  the  maintenance  test  levels. 


2.  Implementation . 

Given  that  the  tasks  of_  functions  V2,  V3,  and  V4  have  been  properly  performed, 
development  of  the  Cl  T  requirements  is  relatively  simplified.  If  there 
has  been  no  prior  interface,  development  of  the  Cl  T  requirements  should 
begin  on  a  qualitative  level.  Conmitments  to  correct  identifiable  short- 
ccmingsjthen  need  to  be  obtained  from  the  design  function.  Development 
of  the  T  requirements  can  then  continue  on  to  a  quantitative  level  for 
inclusion  in  the  Cl  developmert  specifications. 

2.1  Testability  Characteristics  for  Inclusion  in  Specifications.  (35) 

The  following  considerations  should  be  evaluated  for  applicability  to, 
and  inclusion  within,  the  Cl  development  specifications: 


a.  Definition  of  failure  inodes  to  be  used  as  the  basis  for  test  design. 

b.  Fraction  of  faults  detected: 

(1)  percent  of  all  faults  automatically  detected  by  BIT/ETE 

(2)  percent  of  all  faults  detectable  by  BIT/ETE 

(3)  percent  of  all  faults  detectable  on-line  by  BIT/ETE 

(4)  percent  of  all  faults  and  out-of-tolerance  conditions 
detectable  by  BIT/ETE 

(5)  percent  of  all  faults  detectable  by  any  means 

c.  Fraction  of  false  alarms: 

(1)  rate  at  which  false  indications  occur  (per  10®  hours) 

(2)  percent  of  indicated  failures  caused  by  actual  failures 

(3)  percent  of  BIT/ETE  indicated  failures  caused  by  actual  failures 

(4)  percent  of  BIT/ETE  fault  isolations  to  the  wrong  LRU 

d.  Fraction  of  false  status  indications: 

(1)  percent  of  erroneous  BIT  indications 

e.  Requirement  for  number  of  retries  needed  to  declare  a  solid  failure. 

f.  Mean  fault  detection  time: 

(1)  time  to  indicate  a  fault  once  it  has  occurred 

(2)  time  to  detect  a  fault  once  it  has  occurred 

g.  Restrictions  on  built- in-tes.  -  resources  (allocated  from  system 
constraints) . 

h.  Requirements  for  compatibility  (functional,  electrical,  mechanical) 
with  ETE. 

i.  Mean  BIT/ETE  running  time 

(1)  time  to  verify  that  a  failure  has  occurred/or  has  been  repaired 
using  BIT/ETE 

j.  Frequency  of  BIT/ETE  executions 

(1)  time  interval  between  BIT/ETE  executions 

k.  Test  thoroughness 

(1)  percent  of  all  equipment  functions  tested 
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l.  Fault  isolation  resolution 

(1)  isolation  of  P^  percent  of  the  occurred/detected  failures  to  X^ 
UUTs,  P2  percent  of  the  failures  to  X2  UUTs  and  so  on,  with  any 
fault  isolation  method. 

(2)  isolation  of  all  occurred/detected  faults  to  less  than  or  equal 
to  some  maximum  number  of  UUTs. 

(3)  isolation  of  P^  percent  of  the  occurred/detected  failures  to  X^ 
UUTs,  P2  percent  of  the  failures  to  X2  UUTs,  and  so  on,  with 
BIT/ETE. 

(4)  isolation  of  a  specified  percent  of  the  occurred/detected 
failures  to  less  than  or  equal  to  a  specified  quantity  of  UUTs 
at  the  various  maintenance  levels. 

(5)  isolation  of  a  specified  percent  of  the  occurred/detected  to 
less  than  or  equal  to  a  maximum  number  of  plug-in  modules. 

(6)  isolation  semi-automatically  to  a  certain  percent  of  all 
occurred/detected  faults  down  to  a  specified  number  of  UUTs. 

m.  Fraction  of  faults  isolated 

(1)  isolate  a  certain  percent  of  all  failures  that  occur  and/or 
are  detected. 

(2)  isolate  with  BIT/ETE  a  certain  percent  of  all  failures  that 
occur  and/or  are  detected. 

n.  Mean  fault  isolation  time 

(1)  isolate  a  specific  percent  of  failures  that  occur  and/or  are 
detected  within  a  specified  maximum  time. 

(2)  isolate  a  failure  dovn  to  a  replaceable  level,  within  a 
specified  average  time. 

(3)  isolate  a  failure  down  to  a  replaceable  level  withi  a 
specified  time  once  the  fault  isolation  process  has  been 
initiated. 

o.  Requirements  for  test  point  access. 

p.  Test  point  isolation  and  signal  conditioning. 

q.  Test  point  density. 
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r.  Maintenance  personnel  skill  level 

(1)  all  maintenance  actions  must  be  capable  of  being  performed  by  a 
specified  quantity  of  maintenance  personnel  with  a  specified 
skill  level,  at  various  maintenance  levels. 

(2)  BIT/ETE  must  be  designed  for  use  by  a  specified  minimum  skill 
level  technician. 

s.  BIT/ETE  mean-time-to-repair 

(1)  mean-time-to-repair  ETE 

(2)  mean-time-to-repair  monitoring/fault  isolation  functions 

t.  BIT/ETE  mean- time-be tween -failures 

(1)  mean-time-between-fai lures  of  monitoring/fault  isolation  functions 

(2)  mean-time-between-f allures  of  ETE  only 

u .  BIT/ETE  avai labi li ty 

(1)  monitoring/fault  isolations  should  be  operating  with  a  specified 
probability  of  survival 

v.  Mean-time-to-repair 

(1)  system/ equipment  MTTR  and  maximum  repair  time 

(2)  system/equipment  MTTR  and  maximum  repair  time  at  various 
maintenance  levels 

w.  Availability 

(1)  inherent  availability 

(2)  operational  availability 

x.  Active  memory  allocated  for  BIT/ETE  functions 

(1)  monitoring/fault  isolation  functions  shall  take  up  a  specified 
percent  of  active  computer  memory 

y.  Requirements  for  compatibility  with  external  status  monitor. 

z.  Observability  and  controllability  figures  of  merit. 

3.  Completion  Criteria. 

3.1  The  task  is  completed  upon  incorporation  of  the  testability  goals/ 
requirements  of  paragraph  1,  and  acceptance  by  the  government  of  the 
Cl  specifications. 
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PHASE: 


VALIDATION 


FUNCTION: 


Development  Specifications 


TASK  TITLE:  Prepare  Computer  Program  Configuration  Item  (CPC I) 

development  specifications 

task  OBJECTIVE:  Include  testability  with  the  conputer  program  configuration 

item  (CPCI) requirements  in  the  Development  Specifications. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE _ INPUT  TO  T _ OUTPUT  FROM  T 


e  Design 
Engineering 


e  HW/sw  Testability  Features, 
SW  Constraints  Imposed  by 
HW 


e  HW  Testability  Require¬ 
ments  Imposed  by  SW 
Constraints 


e  Application 
Software 


e  SW  Testability  Features 


e  SW  Testability  Require¬ 
ments  Imposed  by  HW 
Constraints 


e  System 
Engineering 


e  System  SW  Testability 
Features 


e  System  SW  Testability 
Requirements  fi  HW 
Constraints 


e  Test 

Engineering 


e  Test  Program  Testability 
Features 


e  Test  Program  SW  Testability 
Requirements  &  Constraints 


COST  TRADEOFF  INTER-RELATIONSHIPS:  None  relevant  to  this  task. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1.1  The  CPCI  specification  defines  Go /No-Go ,  diagnostic  programs,  error 

detection  (interrupt  and  trap  capability),  failure  latency  requirements , 
error  processing  routines  (including  retry,  automatic  error  correction, 
diagnostic  call,  operator  message,  error  logging,  and  immediate  halt)  and 
is  consistent  with  the  FMEA. 

The  CPCI  development  specification (a) ,  after  acceptance  by  the  procuring 
activity,  establish (es)  the  performance  requirements  which  the  CPCI  must 
satisfy  upon  completion  of  the  development  phase.  A  CPCI  developemit 


specification  is  required  for  each  CPCI  allocated  from  the  system  speci¬ 
fication  which  established  the  functional  baseline  or  from  a  higher  level 
configuration  item  or  for  a  non-system  CX.  06)  The  following  guidance 
should  be  applied  to  inputs  to  the  CPCI  specification. (4) 

a.  Computer  Program  Development  Specifications.  The  Computer  Program 
Configuration  Item  (CPCI)  development  specification  for  built-in 
test  software  (GO/NO  GO  and  diagnostic  programs)  and  those  parts 
of  CPCI  development  specifications  for  applications  software  dealing 
with  error  processing  are  based  upon  the  approved  preliminary  design 
data  for  CX  built-in  test. 


b.  Error  Detection.  The  application  software  design  should  include 
sufficient  interrupt  and  trap  capability  to  support  the  immediate 
processing  of  errors  detected  by  concurrent  built-in  test  hardware 
prior  to  the  destruction  of  data  bases  or  loss  of  information  con¬ 
cerning  the  nature  of  the  error.  The  operating  system  and  each 
critical  application  program  should  contain  software  checks 
sufficient  to  meet  failure  latency  requirements. 

c.  Error  Processing.  Error  processing  routines  in  the  application 
software  invoked  by  interrupts  and  traps  should  be  designed  with 

the  full  participation  of  hardware  design  engineers  and  test  engineers. 
The  processing  to  be  performed  (retry,  automatic  error  correction, 
diagnostic  call,  operator  message,  error  logging,  immediate  halt, 
and  others)  should  be  consistent  with  the  failure  modes  and  effects 
analysis.  The  operating  system  hierarchy  should  be  designed  to  allow 
the  diagnostic  software  sufficient  control  and  observation  of  hard¬ 
ware  components. 


3 lamentation. 


Implementation  consists  of  drafting  and  conpleting  the  specifications. 


2.1  The  drafting  of  the  CPCI  specifications  is  consistent  with  MIL-STD-490 
practices  and  relatively  straightforward.  Data  Item  Description  (DID) 
number  DI-E-30139  may  be  used  to  guide  the  writing  of  the  Computer 
Program  Development  Specifications. (36)  The  following  outline,  excerpted 
from  DID  DI-E-30139,  indicates  the  form  and  content: 

1.  General  Requirements 

2.  Detailed  Requirements 

Section  1.  Scope 

1.1  Identification 

1.2  Functional  Summary 


Section  2 

2.1 

2.2 

2.3 

2.4 


Applicable  Docwents 
Program  Definition  Documents 
Inter-Subsystem  Specifications 
Military  Specifications  and  Standards 
Miscellaneous  Documents 


Section  3 

3.1 

3.1.1 

3.1.2 

3.1.3 


Requirements 

Introduction 

General  Description 

Peripheral  Equipment  Identification 

Interface  Identification 


3 . 2  Functional  Description 

3.2.1  Equipment  Descriptions 

3.2.2  Computer  Input/Output  Utilization 

3.2.3  Computer  Interface  Block  Diagram 

3.2.4  Program  Interfaces 

3.2.5  Functional  Description 

3.3  #  Detailed  Functional  Requirements 

3. 3. N  Introduction 

3.3. N.1  Inputs 

3.3. N.2  Processing 

3.3. N.2 (a)  Purpose 

3 . 3 . N. 2 (b)  Functional  Parameters 

3. 3.  N. 2(c)  Diagrams  of  Geometry 

3.3. N.3  Outputs 

3.3. N.4  Special  Requirements 


3 . 4  Adaptation 


3.5  Capacity 


Section  4. 

4.1 

4.2 

4.3 


Quality  Assurance  Provisions 
Introduction 
Test  Requirements 
Acceptance  Test  Requirements 


Section  5  Notes 


Appendix  A 

A.  1 
A. 2 
A. 3 
A. 4 
A. 5 


Mathematical  Analysis 
Mathematical  Derivations 
Alternate  Method 
Summary  of  Equations 
Definitions  of  Terms 
Reference  Documents 


Appendix  B  Miscellaneous  Items 


3.  Completion  Criteria. 

3.1  The  CDHL,  DID,  and  this  compendium  may  be  used  to  assess  completeness  of 
the  CPCI  specifications  in  their  testability  ramifications.  The  task  is 
completed  upon  government  acceptance  of  the  specifications. 

*  N  pertains  to  one  of  a  number  of  functions,  each  indentured  as  shown. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V6C 


PHASE: 


VALIDATION 


FUNCTION: 


Development  Specifications 


TASK  TITLE: 


Select  ETE  or  prepare  ETE  procurement  specification 


TASK  OBJECTIVE:  Reach  a  timely  decision  for  the  selection  of  the  ETE. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER- REIATIONSHIPS : 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


f  . . .  -  "  . . .  . . .  . . . 

e  Test  Equipment 

e  ETE  Specification  | 

Acquisition 

Testability  Require- 

\ 

ments  j 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Not  relevant  to  this  task.  Performing 
this  task  in  context  with  its  predecessor  tasks  in  a  timely  manner  supports 
cost  effectiveness  of  the  overall  program. 


TASK  SYNOPSIS: 


1.  Task  Requirements. 


1.1  The  selection  or  specification  of  ETE  to  support  the  configuration  items 
should  be  accomplished  prior  to  PDR  for  the  prime  configuration  items. 
Units  of  each  configuration  item  which  are  to  be  tested  off-line  (UUTs) 
should  be  identified  during  the  validation  phase.  The  ETE,  whether 
selected  or  developed,  is  to  be  available  during  the  full  scale  develop¬ 
ment  phase  to  support  Test  Program  Set  development,  testability  demonstra¬ 
tion,  and  maintenance  of  the  Engineering  Development  Model.  (5) 


alementation . 


This  task  is  implemented  in  three  steps:  UUT  preliminary  test  require¬ 
ments  analysis,  survey  of  existing  testers  and  using  appropriate  informa¬ 
tion  to  prepare  ETE  procurement  specifications.  Maximum  use  snould  be 
made  of  data  and  information  developed  in  prior  tasks,  notably  those  in 
functions  V3,  V4,  and  V5. 


11-107 


2.1  Test  Requirements  Analysis. 

a.  The  analysis  consists  of  documenting  the  UUT  input-output  signals 
listed  below. 

(1)  Power  supplies  -  ac,  dc. 

(2)  Signal  inputs  (analog)  -  sinusoidal,  pulse,  synchro  and 
resolver,  other  waveforms,  time  delayed. 

(3)  Signal  inputs  (digital)  -  serial  data,  parallel  data. 

(4)  Pressure  input. 

(5)  Signal  outputs  (analog)  -  dc  voltage,  ac  voltage,  phase  angle 
frequency,  time  period,  power  (average  rf ) ,  other  waveforms, 
resistance,  signal  distortion,  synchro  and  resolver. 

(6)  Signal  outputs  (digital)  -  serial,  parallel. 

(7)  Loads  and  networks  -  electronic  elements,  mechanical  elements 

b.  Since  this  task  is  part  of  the  validation  phase,  the  available  UUT 
data  may  be  incomplete  and/or  subject  to  change  during  full  scale 
development.  However,  there  is  strong  reason  to  develop  test  equip 
ment  in  parallel  with  the  prime  system  and  for  the  test  equipment 
to  be  available  during  full  scale  development  test  and  evaluation . 

c.  A  more  complete  checklist  for  requirements  analysis  is  found  in 
M1L-STD-1519 (USAF) ,  Preparation  of  Test  Requirements  Document.  (It 
is  not  a  function  of  this  task  to  prepare  test  requirements  docu¬ 
ments.) 


2.2  Tester  Survey  Methods. 

a.  The  following  areas  should  be  considered  in  documenting  commercial 
ATE  requirements: (37) 

(1)  Required  system-level  capabilities: 

(a)  Maximum 

(b)  Minimum 

(2)  Functions  to  be  measured  on  the  unit  under  test: 

(a)  Equipment  Ranges 

(b)  Accuracies 

(3)  Stimuli  required: 

(a)  Equipment  ranges 

(b)  Accuracies 

(4)  Reliability  and  maintainability  required  of  the  ATE: 

(a)  Desired  mean  time  between  failure 

(b)  Desired  mean  time  to  repair 

(5)  Power  requirements: 

(a)  Facility  available 

(b)  Tester  requirements 
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(6)  Special  features  such  as  environment,  etc. 

(7)  Source  of  software  development  and  estimated  cost. 

(8)  Support  required  for  such  software  as  compilers, 
languages,  programming  manuals,  etc. 

(9)  Maintenance  concept  at  the  requesting  activity  level. 

(10)  Utilization  rate  of  the  ATE  system  (per  day,  month,  year) . 

(11)  Systems  to  be  tested  including  weapons  system  to  be 
supported. 

(12)  Number  of  system  units  to  be  tested  per  year. 

(13)  Testing  times  for  units  under  test  by  types: 

(a)  Using  manual  test  equipment 

(b)  Using  automatic  test  equipment 

(14)  Concurrence  from  applicable  system  manager  and/or  item 
manager  relative  to  the  use  of  ATE  in  support  of  their 
system. 

(15)  Impact  and  cost  of  the  ATE  on  existing  technical  data. 

(16)  Any  specific  equipment  to  be  replaced. 

(17)  Rationale  (facts  and  figures)  used  in  the  cost  analysis. 

(18)  Training  required:  identify  by  operator,  maintenance  and 
programming . 

(19)  Need  date,  with  impact  statement. 

b.  Aid  in  completing  this  task  may  be  found  in  Task  Reference 
Number  6,  specifically  information  concerning: 

e  Tools  and  aids  for  use  in  the  ATE  selection  process  con¬ 
sisting  of  a  list  of  data  banks,  models  and  sources  of 
information 
e  ATE  cost  drivers 

e  ATE  acquisition  factors 


2.3  Select  or  Specify  Criteria. 

a.  When  selecting  PCB  testers  the  usage,  field  or  factory,  should  be 
kept  in  mind.  Table  1  illustrates  the  similarities  as  well  as  the 
differences  in  importance  between  the  two  usages. (38) 

b.  Most  PCS  automatic  test  systems  use  one  of  two  basic  approaches: 
software  simulation  or  hardware  simulation. (39) 

The  advantages  of  software  simulation  include: 

•  The  programmer  has  access  to  an  exact  measure  of  test 
comprehensiveness . 
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TABLE  1.  Factory  vs.  Field:  Importance  of  Test  Scenario  Characteristics 


IMPORTANCE 

TEST  SYSTEM  CHARACTERISTICS 

IMPORTANCE  TO 
FACTORY 

IMPORTANCE  TO 

FIELD  MAINTENANCE 

n 

TESTER  PERFORMANCE 

HIGH 

HIGH 

2. 

FAULT  RESOLUTION  (TO  FAILING 
CHIP  OR  FOIL)* 

HIGH  FOR  FOIL; 

LOW  FOR  CHIPS 

HIGH 

3. 

RELIABILITY  OF  TESTER 

NOMINAL 

HIGH 

4. 

SOFTWARE  DEVELOPMENT 

COST 

NOMINAL, EXCEPT 
HIGH  IN  SMALL 
INEXPENSIVE  PRO¬ 
DUCTS  OF  LOW 
SALES  VOLUME  j 

NOMINAL, EXCEPT 

HIGH  IN  SMALL 
INEXPENSIVE  PRO¬ 
DUCTS  OF  LOW 

SALES  VOLUME 

5. 

TEST  GENERATION  COST 
(MANPOWER  AND  COMPUTER  TIME) 

NOMINAL, EXCEPT 
HIGH  IN  SMALL  < 

INEXPENSIVE 

PRODUCTS  OF  LOW 

SALES  VOLUME 

NOMINAL, EXCEPT 

HIGH  IN  SMALL 
INEXPENSIVE 

PRODUCTS  OF  LOW 
SALES  VOLUME 

6. 

EASE  OF  TEST  UPDATE 

HIGH 

HIGH 

7. 

TESTER  REPRODUCTION  COST 

NOMINAL 

NO!  INAL 

8. 

TEST  DATA  BASE  COST 

LOW 

HIGH 

9.  UNIT-UNDER-TEST  LOGIC  DESIGN 
GUIDELINES  CONSTRAINTS 

j 

DESIRABLE, BUT 
DIFFICULT  TO 
IMPOSE  ON 
ENGINEERING 
WITHOUT  HIGHER 
MANAGEMENT 
DIRECTIVES 

DESIRABLE, BUT 
DIFFICULT  TO 
IMPOSE  ON 
ENGINEERING 

WITHOUT  HIGHER 

MANAGEMENT 

DIRECTIVES 

10.  ABILITY  TO  TEST  AT  HIGH 

HIGH 

HIGH 

‘FOIL:  THE  PCB  CONDUCTING  PATHS  FROM  CHIP  TO  CHIP  AND  I/O  TO  CHIP 


•  Except  for  final  verification,  program  preparation  does  not 
require  a  known-good  PC  board. 

e  Automatic  test  generation  software  is, available. 

Disadvantages  of  software  simulation  include: 

e  Modeling  components  not  in  the  ATE's  library  can  prove  time- 
consuming  and  costly. 

e  Simulation  runs  take  a  lot  of  time  (but  often  can  run 
unattended. 

e  PC  board  programming  proves  expensive  overall,  and  the  user 
forever  depends  on  the  ATE  manufacturer  for  successive 
library  updates. 

The  advantages  of  hardware  simulation  include: 
e  Programming  does  not  require  a  description  of  each  inter¬ 
connection  on  the  board, 
e  No  component  modeling  is  needed, 
e  Programming  proves  less  expensive. 

The  disadvantages  include: 

e  The  UBer  has  no  precise  measure  of  test  program  comprehensive¬ 
ness. 

•  Program  preparation  does  require  a  known-good  PC  board, 
e  No  automatic  test  generation  is  available. 

c.  The  general  purpose  tester,  such  as  the  GR2225,  is  a  cost  effective 
microprocessor  testing  solution  based  on  the  following  criteria: (40) 

e  The  user  can  characterize  the  microprocessor  to  qualify  and 
monitor  vendor  quality. 

e  High  level  programming  permits  easier  programming  and 
debugging. 

•  Information  is  available  for  management  reports. 

e  Test  integrity  is  maintained  since  operational  uncertainties 

are  eliminated. 

e  Correlation  exists  between  field  and  factory  testing. 

d.  The  complexity  of  the  new  generation  of  ATE  requires  a  system 
approach  to  calibration  (i.e.,  the  incorporation  of  calibration 
standards  as  an  integral  part  of  the  system) .  The  major  advantages 
of  this  method  acre:  (41) 

e  Instruments  are  not  removed  from  the  system, 
e  Since  it  is  done  with  programing,  human  errors  are  reduced, 

e  The  time  required  to  run  the  calibration  program  is  consider¬ 

ably  less  than  the  actual  calibration  time  required  for 
individual  instrument. 

e  When  the  built-in  standards  are  sent  out  for  calibration,  ATE 
can  still  be  used  for  UUT  testing, 
e  The  system  is  fully  calibrated. 
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•  The  system  is  easily  adaptable  to  change  due  to  IEEE  488-1975 
bus. 

e  There  is  reduction  in  Mean  Time  to  Repair  (MTTR)  due  to 
automatic  programs . 


e.  As  the  complexity  and  costs  of  manufacturing  PCBs  increase,  develop¬ 
ment  of  a  comprehensive  board  test  fixture  effort  involves  tradeoffs 
also.  (42)' 


(1)  Manually  operated  test  fixtures  are  the  leapt  expensive  to 
acquire,  but  require  more  time  and  effort  to  operate  and 
generally  have  a  limited  number  of  probes. 

(2'  Vacuum  fixtures  can  be  acquired  at  reasonable  cost,  are  easy 
to  operate,  and  provide  full  access  to  the  top  of  the  board, 
but  seal  li->  expectancy  and  vacuum  flow  requirements  may  be 
adverse  factors. 

(3)  Pneumatically  operated  fixtures  are  most  costly  to  acquire, 

but  overcome  large  probe  force  and  test  boards  with  components 
on  both  sides. 

f.  When  selecting  ATE,  consideration  should  be  given  to  the  use  of 

Graphic  Display  Terminals  (GDT)  as  a  fault  isolation  aid.  Unique 

features  and  benefits  of  such  a  maintenance  method  are: <43) 

e  The  GDT  need  not  be  wired  to  the  ATE.  It  could  be  used  to 

support  many  different  ATEs  in  the  same  facility.  Only  the 
program  tape  cartridges  are  peculiar  to  the  ATE. 

e  The  GDT  can  also  be  used  as  a  training  aid  to  pictorially  dis¬ 
play  controls,  switches  and  functions  on  the  ATE  for  completely 
unfamiliar  personnel. 

e  The  GDT  can  be  used  to  assist  system  programmers,  operator  and 
even  calibrators  in  their  specific  areas  of  responsibilities. 

e  All  symptoms  of  failures  and  the  resulting  corrective  actions 
could  be  written  onto  a  tape  cartridge.  Recall  by  similar 
symptoms  would  list  all  previous  corrective  actions.  Such 
historical  data  can  also  be  consolidated  and  interchanged 
between  different  facilities  using  the  same  ATE. 


2.4  Formulating  ATE  Specifications. 

a.  Help  in  formulating  the  specification  requirements  may  be  found  in 
Task  Reference  Number  V6A,  Cl  Development  Specifications.  ATE 
specifications  should  be  prepared  per  MIL-STD-490,  Specification 
Practices. 
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b.  The  following  is  a  checklist  to  prepare  specifications  for  digital 

test  equipment: 

Determine  and  identify: 

(1)  Level  of  assembly  to  be  tested. 

(2)  Method  of  testing  to  be  used. 

(3)  Types  of  digital  signals  to  be  tested. 

(4)  The  types  of  digital  tests  to  be  made. 

(5)  The  following  characteristics  should  then  be  identified  and 
defined: 

(a)  For  Parallel  Signals: 

•  Number  of  simultaneous  signal  input  and  respo:  se 
output  lines  available. 

•  Maximum  and  minimum  rate  that  input/  both  repeated 
input  and  changed  input,  can  be  applied  and  output 
evaluated. 

•  Input  and  output  data  formatting  characteristics. 

•  Data  storage  capabilities. 

•  Control,  conditioning,  and  clock  signals  available 
and  their  characteristics. 

•  Stimuli  characteristics  and  control,  and  response  kit 
characteristic  evaluation  capabilities. 

•  Logic  family  capabilities. 

•  Input  and  output  impedance  characteristics. 

(b)  For  Serial  Signals: 

•  Stimuli  characteristics  including  word  lengths  and 
control,  bit  rates  and  control,  and  word  transfer 
characteristics . 

•  Data  storage  capabilities. 

•  Available  control,  conditioning,  and  clock  signals 
and  their  characteristics. 

•  Logic  family  capabilities. 

'  -  Input  and  output  inpedance  characteristics. 

•  Response  evaluation  characteristic  of  word  lengths, 

/  >  bit  characteristics  and  word  transfer. 


Completion  Criteria. 

The  selection  of  test  equipment  is  based  upon  its  ability  to  meet  UUT 
test  requirements,  its  ability  to  be  maintained  and  its  effect  upon  ICC. 
The  task  is  completed  upon  selection  of  the  test  equipment  (preferably 
in  government  inventory)  and/or  approval  of  the  Development  Specification, 
Product  Specification  or  Inventory  Item  Specification. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  V7A 


PHASE: 

FUNCTION: 

TASK  TITLE: 

TASK  OBJECTIVE: 


VALIDATION 
PDR  Support 
Support  PDR 

Assure  that  Testability  is  properly  represented  and 
evaluated  at  the  PDR. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS t 

Coordination  is  appropriate  with  all  other  disciplines  participating  in  the 
PDR.  The  inter-relationships  are  those  of  the  functions  and  tasks  which 
precede  the  PDR. 


COST  TRADEOFF  INTER-RELATIONSHIPS i  Not  applicable  to  this  task.  The 
underlying  inter-relationships  are  those  of  the  functions  and  tasks  which 
precede  the  PDR. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

At  the  Preliminary  Design  Review  (PDR) : 

(1)  Present  the  qualitative  end  quantitative  testability  predictions 
with  support  analysis  (including  tradeoffs,  LCC  and  reliability  data) ; 

(2)  Present  status  of  conpliance  of  prime  system  design  with  testability 
requirements; 

(3)  Review,  revise  and/or  approve  prime  system  development  specifications. 

1.1  The  review,  revision,  and  approval  of  development  specifications  is  to  be 
accomplished  through  the  Preliminary  Design  Review  (PDR)  process  per 
MIL-STD-1521.  The  testability  design  tradeoff  analysis  should  be  docu¬ 
mented  and  requirements  for  detail  design  presented  at  the  PDR.  Design 
engineers  should  present  qualitative  and  quantitative  predictions  of 
testability,  with  supporting  analysis,  and  identify  opportunities  for 
further  enhancement  of  testability  through  tradeoffs  with  performance, 
cost,  reliability,  etc.  In  choosing  between  alternate  designs,  prefer¬ 
ence  must  be  given  to  the  simplest  design  which  meets  the  testability 
requirements. (5) 


2. 


Implementation . 

2.1  Preparation  for  the  PDR  consists  of  review  of  all  generated  testability 
materials  and  review  of  the  development  specifications,  followed  by 
organization  of  the  major  testability  elements  for  concise,  unambiguous 
presentation. 

Specifications  for  the  Preliminary  Design  Review  process  are  contained 
in  MIL-STD-1521(USAF) ,  Technical  Reviews  and  Audits  for  Systems, 

Equipment  and  Computer  Programs. 

The  testability  engineer  may  present  summary  information  from  the 
Testability  Analysis  Report  with  all  necessary  supporting  data.  This 
should  include  tradeoff  results,  modeling  results,  partitioning  analysis, 
observability  analysis,  controllability  analysis,  initialization  analysis, 
and  BIT/ETE  analysis. 

The  testability  engineer  must  also  participate  in  the  review,  revision, 
and  approval  of  the  development  specifications  at  PDR. 


2.2  The  PDR  may  result  in  corrective  or  other  actions  being  assigned  which 
impact  on  testability.  Responsibility  for  these  actions  will  also  be 
assigned.  It  is  appropriate  for  the  Testability  discipline,  while 
active  in  the  developmental  program ,  to  be  cognizant  of  actions  tar  3 
their  status)  assigned  to  other  disciplines  or  managers. 


3.  Completion  Criteria. 

Government  approval  of  the  Testability  Analysis  Report  and  the  equipment/ 
development  specifications,  together  with  approval  of  PDR  minutes  and 
completion  of  action  items  related  to  testability  comprise  the  measure 
of  completion. 
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Full  Scale  Development  Phase  Testability  Tasks 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F1A 

PHASE:  FULL  SCALE  DEVELOPMENT 

FUNCTION:  Test  Requirements  Analysis  Performance 

TASK  TITLE:  Produce  a  validated  Test  Requirements  Document  (TRD) 

for  each  UUT 

TASK  OBJECTIVE:  The  objective  of  this  task  is  to  create  and  validate  a 

document  that  serves  as  a  single  source  of  all  performance 
verification  and  diagnostic  procedures  and  for  all  equipment  requirements  to 
support  the  UUT  in  its  maintenance  environment  independent  of  any  specific 
test  apparatus. 


DESIGN  DISCIPLINE  TESTABILITY  (T?  INTER-RELATIONSHIPS: 


T  INTERFACE  INPUT  TO  T 

OUTPUT  FROM  T 

e  System  Engineer-  e  Design  and  Performance 

•  TRD 

ing  Design 

Engineers 

e  Maintainability 

e  T  Analysis;  TRD 

Engineering 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Not  specifically  applicable  to  this 

task.  The  previous  testability  design  effort  simplifies  the  diagnostics  and 
subsequent  test  program  set. 


TASK  SYNOPSIS: 
l.  Task  Requirements. 

1. 1  Test  Requirements  Analysis. 

A  Test  Requirements  Analysis  defines  the  functional  end-to-end  (perform¬ 
ance)  test  requirements  and  fault  isolation  test  requirements  for  each 
item.  The  input  to  the  analysis  process  is  Cl  and  UUT  design  data  con¬ 
sisting  of  drawings  (schematics,  logic  diagrams,  parts  lists,  etc.), 
failure  data,  performance  specification,  theory  of  operation,  mechanical/ 
electrical  interface  definition,  and  testability  analysis  data. 

1.2  UUT  Test  Requirements  Document. ^ 

The  Test  Requirements  Document  (TRD)  should  constitute  the  formal  inter-  > 
face  between  the  activity  responsible  for  detailed  hardware  design  and 
the  activity  responsible  for  Test  Program  Set  development  per  MIL-STD- 
1519 (USAF).  This  document  serves  as  a  single  source  of  all  performance 
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verification  and  diagnostic  procedures,  and  for  all  equipment  require¬ 
ments  to  support  the  UUT  in  its  maintenance  environment,  whether 
supported  manually  or  by  ATE  or  ETE. 

The  trd  provides  detailed  configuration  identification  for  UUT  design  and 
test  requirements  data  to  ensure  compatible  test  programs.  The  test¬ 
ability  analysis  performed  during  the  validation  phase,  refined  during 
the  full  scale  development  phase,  and  documented  in  the  Testability 
Analysis  Report  can  be  used  as  a  partial  basis  for  the  Test  Requirements 
Document  for  each  UUT. 

1. 3  Validation. 

The  data  in  the  TRD  should  be  validated  by  actual  measurements  made  on 
the  UUT. 


2.  Implementation . 

2.1  The  following  summarizes  the  needs  to  be  addressed  by  the  data  assembled 
in  a  TRD. <2> 

a-  Deployed  Equipments. 

(1)  UUT  performance  verification  programs/procedures  to  provide  a 
screening  capability  for  0  or  I  level  maintenance  activities. 

(2)  UUT  fault  isolation  program/procedures  to  provide  a  complete 
diagnostic  capability  to  component  level  for  0,  I,  or  D 
maintenance  levels. 

(3)  Identification  of  candidate  ATE  systems  based  on  the  UTT  test 
requirements . 

(4)  UUT  source  and  test  Requirements  Supplemental  Data  to  support 
program/procedures  during  operational  deployment  and  to  provide 
a  capability  for: 

(a)  Program/procedure  debugging  and  modifications 

(b)  Configuration  control  of  support  proqram/procedures 

(c)  On-line  troubleshooting  of  UUT  and  test  program 
problems  under  deployment  conditions 

b.  Pre-Deployment . 

The  need  for  adequate  test  requirements  documentation  extends  to  the 
preproduction  and  development  phases. 

Performance  specifications  for  the  prime  equipment  are  being  refined 
during  the  development  process  together  with  the  other  s-arce  data 
that  will  be  required  for  the  TRD.  Examples  of  such  data  are  sche¬ 
matics,  logic  diagrams,  family  tree  (configuration),  parts  lists. 
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c.  Standardization  of  Data 

In  order  to  meet  the  objectives,  the  quality  and  standardization  of 

the  TRD  must  be  ensured  for  the  purpose  of: 

(1)  Providing  released  UUT  source  data  of  clearly  identified 
configuration . 

(2)  Providing  clear  traceability  of  UUT  test  requirements  to 
the  UUT  source  data. 

(3)  Providing  ease  of  UUT  data  storage  and  retrieval  through 
standardized  organization  of  the  data. 


2.2  Unless  otherwise  specified,  the  Test  Requirements  Document  may  be  prepared 
in  accordance  with  MIL-STD-1519(USAF) . The  TRD  purpose  is  to  provide 
the  information  necessary  to  test  the  UUT  in  the  most  efficient  manner 
possible  and  with  a  minimum  of  UUT  interface  while  verifying  all  required 
performance  characteristics. 

The  TRD  may  be  initially  prepared  ;o  reflect  the  preproduction  model  con-, 
figuration  of  the  UUT.  This  version  of  the  TRD  is  completed  when  the 
configuration  of  the  first  preproduction  model  is  baselined.  The  TRD  is 
then  revised  to  reflect  the  production  configuration  when  the  production 
model  is  baselined. 


Contents  of  the  TRD  include  the  following  itens: 


a.  Cover  Sheet 

b.  Approval  Sheet 

c.  Revision  Index  Sheet 

d.  Configuration  Data 

e.  General  Data 

f.  UUT  Interface  Requirements 
rj.  Detailed  Performance 

Characteristics 


h.  Detailed  Test  Information 

i.  Outline  Installation  Drawings 

j .  Unit  (Main)  Assembly  Drawings 

k.  Detail  &  Subassembly  Drawings 

l.  Wiring  Drawings 

m.  Functional  Block  Drawings 

n.  Test  Flow  Chart 


Each  of  t\ese  ite~~s  is  described  in  detail  within  the  MIL-STD  as  to  the 
specifics  of  complete  documentation.  For  the  first  seven  items  the 
appendix  to  MIL-STD-1519  provides  the  required  format  for  submittal.  By 
far  the  bulk  of  the  TRD  original  work  is  the  detailed  test  information. 
Each  test  to  be  conducted  on  the  UUT  is  detailed  on  one  separate  test 
information  sheet.  The  data  for  each  test  completely  describes  all  input 
conditions  and  the  measurements  required  to  perform  the  test,  and  the 
I/O  connections  specified  by  con-  actor  and  pin  number. 


The  TRD  is  validated  by  applying  the  TRD  specified  I/O  to  a  certified 
gcod  UUT  and  verifying  that  the  TRD  values  are  obtained. 
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2.3  Each  test  in  the  TRD  should  be  validated  by  the  contractor.  Validation 
will  be  accomplished  by  applying  the  inputs,  loads,  etc.,  specified  by 
the  TRD  to  an  acceptable  (a  certified  good)  UUT  and  verifying  that  the 
specified  values  are  obtained.  A  validation  certificate  can  be  provided 
with  each  TRD.  The  validation  certificate  includes  the  following 
informations 

a.  A  listing  of  test  numbers,  with  the  actual  values  obtained  from  the 
measurements  made  during  validation  testing. 

b.  Complete  listing  or  identification  of: 

(1)  TRD 

(2)  UUT 

( 3 )  Test  equipment 

(4)  Test  personnel 

(5)  Contract  number 

(6)  Supplier 

(7)  Sub-supplier  (where  applicable) 

c.  Date  testing  was  accomplished. 

d.  Signature  of  test  persornel. 

e.  Signature  of  the  procuring  activity  representative  who  witnessed  or 
participated  in  the  testing. 


Completion  Criteria. 


3.1  This  task  is  completed  upon  acceptance  of  the  TRD  by  the  processing 
activity.  Acceptance  of  the  data  required  by  this  specification 
accomplished  by  submittal  of  copy  of  the  validated  TRD  and  validation 
certificate  to  the  procuring  activity.  This  acceptance,  however,  is 
contingent  on  final  review  of  the  delivered  materials  by  the  procuring 
activity.  The  procuring  activity  notifies  the  trd  contractor  of  final 
acceptance  of  the  data  required  by  this  specification. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  FIB 


PHASE; 


Full  Scale  Development 


FUNCTION: 


Test  Requirements  Analysis 


TASK  TITLE: 


Produce  a  diagnostic  software  specification 


TASK  OBJECTIVE:  Create  a  document  that  serves  as  a  single  source  of  all 

diagnostic  software  tests  for  each  UUT. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


Application 

Software 


•  T  Analysis 


COST  TRADEOFF  INTER-RELATIONSHIPS :  Not  relevant  to  this  task. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

(4) 

1.1  General  Requirements. 

The  diagnostic  software  specification  describes  in  detail  all  the 
operational  and  functional  requirements  necessary  to  design,  test,  and 
maintain  the  required  computer  program.  In  addition,  it  provides  the 
logical,  detailed  descriptions  of  the  performance  requirements  of  a 
digital  computer  program.  The  requirements  stated  in  the  specification 
are  compatible  with  all  componi  .ts  of  the  digital  system  and  inter¬ 
faced  systems.  However,  the  specification  should  not  unnecessarily 
duplicate  descriptive  material  presented  in  other  documents. 

1.2  Use  may  be  made  of  the  testability  analysis  performed  during  the 
Validation  phase  and  documented  in  the  Testability  Analysis  Report  as 
a  partial  basi3  for  the  specification  of  diagnostic  software.'*' 

Use  may  also  be  made  (to  the  extent  available)  of  the  test  data 
developed  at  the  UUT  level  (for  external  testing)  in  designing 
diagnostic  software  tests.  If  needs  arise  to  revise  the  specification, 
updated  FSD  phase  data  should  be  incorporated  or  reflected  as 
appropriate . 


2,  Implementation . 

2,1  The  following  outline  is  a  normal  format  for  presentation  of  the 
material  in  the  Diagnostic  Software  Specification;' 

a .  Scope 

(11  identification  of  System  and  Software  Content 
(21  Functional  Summary  of  System  and  Software 

b.  Applicable  Documents 

(.11  Program  Definition  Documents 
(2)  Inter-Subsystem  Specifications 
(31  Military  Specifications 
(4}  Miscellaneous  Documents 

c.  Requirements 

(11  introduction 

(a)  General  System  Description 

(b)  Peripheral  Equipment  Identification 
(cl  Interface  Identification 

(21  Functional  Description 

(al  Equipment  Descriptions 

(b)  Computer  Input/Output  Utilization 

(cl  Computer  Interface  Block  Diagram 

(dl  Test  Language  and  Compiler  Designation 

(el  Program  Interfacts 

(f 1  Functional  Description 

(3)  Detailed  Functional  Requirements 

(al  Introduction 

1^  Inputs 
2,  Processing 

a.  Purpose 

b.  Functional  Parameters 

c.  Diagrams  of  Geometry 

2.  Outputs 

_U  Special  Requirements 
(41  Adaptation 
(5)  Capacity 
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d.  Quality  Assurance  Provisions 


(!)  Introduction 

(a)  Computer  subprogram  testing 
(bl  Computer  program  testing 
(c)  Computer  program  acceptance  testing 
(dl  System  Integration  Testing 

(.2)  Test  Reqi  \rements 

(.3)  Acceptance  Test  Requirements 

2.2  Special  attention  should  be  given  to  computer  language  used  in 
automatic  testing.  ATE  languages  provide  the  vehicle  for  expressing, 
modeling  and  solving  test  problems.  Bibliography  reference  number  5 
focuses  on  the  nature  of  different  types  of  test  languages,  the  roles 
they  play  in  testing  and  the  qualities  that  make  them  useful  or 
difficult  to  implement.  The  paper  discusses: 

a.  ATE  Software  Components:  support  software,  control  software,  test 
application  software  and  UUT  resident  software. 

b.  Test  procedures  vs.  test  programs. 

c.  Special  purpose  languages 

d.  Language  levels  and  trends 

e.  Support  software  costs 

f.  Language  cost  considerations 

Certain  high-level  languages  provide  better  visibility  to  managers. 
Natural  engineering  syntax  and  vocabulary  improve  communications 
between  the  test  programmers  and  managers. 

2.3  The  CDRL,  DID  and  this  compendium  may  be  used  to  assess  completeness 
of  the  testability  content  of  diagnostic  software  specification. 

3.  Completion  Criteria. 

3.1  The  task  is  completed  upon  government  acceptance  and  approval  of  the 
specifications . 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  Pic 


PHASE;  Full  Scale  Development 

FUNCTION;  Test  Requirements  Analysis 

TASK  TITLE;  Test  Program  Set  (TPS)  Development 

TASK  OBJECTIVE;  Assure  that  each  UUT  is  properly  supported  by  a  valid 

and  complete  Test  Program  Set. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS; 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  TPS  Design 

e  ID  Description,  Drawings 

•  T  Requirements,  Hardware 

Engineering 

Review 

e  TPS  Software 

e  SW  Flow  Diagrams, 

e  T  SW  Requirements, 

Engineering 

Listing 

Review 

e  Program  Manager 

•  T  Review  Comments  for 

TPS 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Analysis  is  required  to  optimize  the 

range  and  depth  of  the  test  program  sets 
and  to  determine  the  TPS  approach  which  minimizes  the  net  costs  of  TPS 
acquisition  and  the  costs  of  manpower  and  other  support  resources  used  in 
conjunction  with  the  TPS  in  the  deployed  phase. 

TASK  SYNOPSIS: 

1.  Task  Requirements ; 

Every  unit  to  be  tested  needs  an  engineered  interface  with  its  test 
device.  Modem  practice  emphasizes  the  TPS  as  an  interface  with 
Automatic  Test  Equipment.  Some  similar  form  of  interface,  perhaps 
lesser  scope  than  a  full  TPS,  is  also  required  if  the  test  device  is 
other  than  ATE.  This  task  is  initiated  ahead  of  critical  design 
review  so  that  the  TPS  approach  may  be  reviewed  there.  The  TPS 
themselves  are  developed  later  in  FSD  and  fabricated  during  the 
production  phase. 


1. 1  Test  Program  Set. 

A  TPS  is  to  be  prepared  for  each  UUT.  A  TPS  is  composed  of  one  test 
program  (TP) ,  one  interface  device  (ID) ,  one  Test  Program  Instruction 
(TPI) ,  document,  and  supplementary  data.  Although  an  ID  may  be 
shared  by  multiple  UUTs ,  the  TP,  TPI,  and  supplementary  data  are 
unique  to  a  UUT  and  its  multiple  configurations. '  ' 

1 . 2  Development  and  Delivery. 

TPS  scope  includes  development,  test,  quality  assurance  and  configura¬ 
tion  management  of  the  family  of  TPS.  Production  monitoring  is 
included  in  Task  P1A. 

2.  Implementation . 

MIL-STD-2077 (AS) ^  establishes  the  requirements  for  the  development, 
test  documentation,  configuration  management,  quality  assurance,  and 
preparation  for  delivery  of  Test  Programs  (TPs)  and  that  related 
hardware  and  documentation  to  be  used  in  conjunction  with  an  appro* 
pri  te  Automatic  Test  Equipment  (ATE)  to  test  Units  Under  Test  (UUTs) . 

The  following  implementation  guidance  is  adapted  from  MIL-STD-2077 
and  other  sources  as  referenced.  The  appendix  provides  some  detailed 
technical  guidance. 

2.1  Teat  Program  Set  Contents. 

The  content  requirements  for  the  TP,  TPI,  supplementary  data,  and  ID 
are  as  follows: 

a.  Test  Program  Content.  The  TP  contains  a  coded  sequence  which, 
when  executed  by  the  ATE,  provides  the  system  a  set  of  instruc¬ 
tions.  It  consists  of: 

•  Program  Heading  and  Identification 

•  Self-Test  Survey 

•  Identity  Checks 

•  Safe-to-Tum-On  Tests 

e  Performare  Routines  (end-to-end  test) 
e  Diagnostic  Fault  Isolation  Routines 

•  Program  Entry  Points 

b.  Test  Program  Instruction.  The  contents  of  the  TPI  contain  that 
information  which  cannot  be  communicated  by  the  ATE  under 
control  of  the  TP  (hook-up,  probe  point  locations...)  and  is 
required  to  accomplish  testing  of  the  particular  UUT  such  as 
pretesting  data,  test  data  and  post-testing  data. 
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c.  Supplementary  Data.  The  contents  of  the  supplementary  data 
contain  that  information  necessary  to  maintain  and/or  modify  the 
TPS  and  analyze  the  TPS  and  UUT  in  case  of  a  problem  or  anomaly 
during  testing.  It  includes  all  that  additional  information 
essential  to  a  full  comprehension  of  the  intent,  structure  and 
interrelation  of  all  elements  of  the  TPS. 

d.  Interface  Device.  The  ID  provides  the  mechanical  and  electrical 
connection  and  signal  conditioning,  if  required,  between  the 

ATE  and  the  UUT.  It  is  a  requirement  to  minimize  the  complexity 
of  IDs  subject  to  the  following  rules: 

(1)  Optimize  IDs  so  that  as  many  UUTs  as  practical  can  be  tested 
by  the  same  basic  ID  assembly,  with  the  objective  of  reducing 
the  total  number  of  IDs  required  and  thus  reducing  shop 
storage  requirements.  The  IDs  should  be  designed  with  a  20% 
expansion  capability.  That  is,  provisions  are  made  in  the 
design  of  the  ID  to  accommodate  unanticipated  ID  requirements 
including  number  of  wires ,  added  functions  and/or  subassem¬ 
blies  20%  greater  than  the  defined  requirements. 

(2)  Each  ID  will  have  a  minimum  Mean  Time  Between  Failure  (MTBF) 
of  1000  hours  calculated  in  accordance  with  MIL-HDBK-217. 

(3)  Each  ID  will  be  small  enough  to  permit  both  the  ID  and  any 
UUT  to  be  physically  supported  by  the  intended  ATE  work 
surface . 

(4)  IDs  are  designed  in  conformance  with  the  requirements  of 
MIL-T-288 00  type  III  class  4  equipment. 

(7) 

2.2  Test  Program  Set  Development. 

The  overall  process  of  developing  a  TPS  is  shown  in  Figure  1.  The 
Test  Requirements  Analysis ,  Task  F1A  provides  the  foundation  for 
TPS  development. 

a.  Test  Program  Specification  Phase.  The  first  task  is  to  develop 
the  Test  Program  Specification,  starting  with  a  functional 
flow  chert  (for  the  tests)  and  culminating  in  an  English  Language 
Test  Document  (ELTD)  which  contains: 

•  A  brief  narrative  description  of  all  go-chain  tests 

•  A  flow  chart  showing  the  go-chain  and  each  no-go  chain  (al 1 
stimulus  and  measurement  functions  are  identified) 

•  A  statement  of  all  ATE  operator  instructions 

•  Specific  identification  of  the  test  adapter 


TEST 

REQUIREMENTS 

ANALYSIS 
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PROGRAM 

SPECIFICATION 


PROQR AM 
DESIGN  ANO 
PRODUCTION 


•  DATA  COLLECTION 

•  FLOWCHART 

•  GENERATE 

•  DATA  ANALYSIS 

•  DEFINE  TESTS 

COMPLETE  TEST 

•  REIVEW 

•  DESIGN  ADAPTER 

PROGRAM 

STIMULUS  ANO 

•  ELTO  DOCUMENT 

•  BUILD  ADAPTER 

MEASUREMENTS 

•  DESIGN  REVIEW 

•  CODE 

•  TESTS 

•  TEST  PLAN 
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•  PERFORMANCE 

•  COMPILE 

•  DIAGNOSTICS 

•  EDIT 

•  ADAPTER  DEFINITION 

•  COST  ESTIMATE 
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SET 
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DELIVERY 


«  VALIDATE 

•  DEMONSTRATION 

PERFORMANCE 

•  SELL-OFF 

•  VALIDATE 

•  ACCEPTANCE  TEST  REPORT 

DIAGNOSTICS 

•  VALIDATE 

SELF  TESTS 

•  Fault 

INSERTION 

OTHER  CONSIDERATIONS 

ATE  MAINTENANCE 

UUT  MAINTENANCE 

OEM  CONSULTATION 

ATPG  SUBCONTRACTS 

CONFIGURATION  MANAGEMENT 

Figure  1.  Test  Program  Set  Development 
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During  this  specification  phase,  all  stimulus,  measurements,  and 
calculations  are  identified  with  their  appropriate  tolerances. 

All  go-chain  tests  and  alignment  tests  are  identified.  No-go 
chains  and  their  respective  "condemned"  subassemblies  or  parts 
are  identified. 

It  is  appropriate  to  conduct  a  design  review  to  include  customer 
and  original  equipment  manufacturer  personnel,  with  goals  as 
follows : 

e  Verify  the  validity  of  the  test  approaches 

•  Answer  questions  from  the  TPS  designer(s) 

•  Establish  ground  rules  for  selection  of  UUT  faults  to  be  used 
in  validation 

Program  Design  and  Production  Phase.  The  task  is  to  expand  the 
functional  tests  into  their  respective  detailed  tests,  including 
all  stimulus  and  measurements  techniques,  to  yield  a  complete 
test  program.  At  this  point,  all  data  required  to  release  the 
build  of  the  test  adapter  is  known,  hence  it  should  be  released 
for  build.  The  detailed  tests  previously  generated  are  coded  into 
the  appropriate  high  order  language  statements  which  are  input  to 
the  operating  system  for  compilation.  This  initial  compilation 
is  edited  and  a  listing  is  generated  which  corresponds  to  the 
baseline  test  program  flowchart.  The  combination  of  the  test 
flowchart,  the  test  adapter  description,  and  the  listing  form  the 
initial  ELTD  to  be  updated  throughout  the  remaining  validation 
and  acceptance  phase. 

Program  Validation  Phase. 

(1)  First  the  test  adapters  are  connected  to  the  ATE  and,  using  a 
"validation"  mode  of  operation,  the  test  program  is  executed 
without  the  UUT.  This  is  done  to  verify  that  stimulus  appears 
at  designated  interface  points ,  hence  provides  protection  for 
the  UUT.  Next  the  UUT  is  connected  to  the  ATE.  Each  go-chain 
test  is  executed  until  certified  as  correct.  Verifications 
are  accomplished  on: 

•  Performance  Limits 

•  Timing 

•  Operator  Instructions 

•  Adjustment  Routines 

Test  requirements  which  surface  and  which  were  not  included 
in  the  initial  ELTD,  but  which  are  obviously  necessary ,  are 
added  as  needed  during  this  part  of  the  validation.  When 
the  go-chain  tests  are  completely  validated  and  with  a  good 


UUT  connected  to  the  ATE,  the  program  is  forced  through 
each  no-go  chain  using  the  "system  validation"  mode  of 
operation.  This  is  done  in  order  to  verify  coding,  operator 
interaction,  and  printouts.  Absolute  test  diagnostic 
capability  is  still  not  validated  at  this  point  sinco 
the  no-go  condition  was  "forced"  while  the  UUT  did  not  in  fact 
harbor  a  fault. 

(21  Next  in  the  validation  phase  comes  fault  insertion.  While 
this  is  strictly  an  empirical  effort,  it  remains  the  most 
important  method  of  assuring  a  quality  program.  It  is 
accomplished  by  inducing  faults,  one  at  a  time,  into  the  UUT. 
With  each  fault,  the  program  is  executed  to  validate  the 
diagnostic  capability  of  the  program.  Whenever  the  program 
fails  to  detect  or  correctly  isolate  a  given  fault,  further 
analysis  is  required.  In  all  cases  it  is  necessary  to  confirm 
that  all  selected  faults  do  in  fact  drive  one  or  more  of  the 
UUT 1 s  operating  parameters  beyond  specified  tolerances. 

d.  Program  Set  Acceptance.  The  Test  Program  Set  acceptance  test 

usually  follows  a  procedure  such  as  this : 

•  A  final  ELTD  is  submitted  to  the  customer  in  sufficient  time  for 
him  to  review  the  program  and  select  the  faults  for  acceptance 
test, 

•  On  acceptance  day,  the  test  program  is  loaded  into  the  ATE  in 
source  language  and  translated  by  the  compiler. 

•  A  good  UUT  is  tested  to  show  that  the  program  recognizes  a  UUT 
that  functions  within  specified  tolerances. 

•  Faults,  one  at  a  time,  are  induced  into  the  UUT.  These  faults 
are  those  previously  selected  by  a  customer  representative. 

•  If  faults  are  incorrectly  isolated,  the  program  is  corrected  and 
re-run, 

•  Upon  successful  completion.  Acceptance  Test  Records  are 
accumulated  and  witnessed  by  all  interested  parties. 

Within  30  (or  otherwise  specified)  days  after  the  acceptance  test, 

the  final  delivery  is  made  and  should  include: 

•  Acceptance  Test  Report 

•  Source  and  Object  Programs  on  magnetic  tape 

•  Final  Test  Accessory 

•  Final  ELTD 
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2.3  Configuration  Management  and  Quality  Assurance. 

a.  The  configuration  of  the  TPS  is  managed  by  documentation  and  use  of 
procedures  for  the  identification,  control,  updating  and  status 
accounting  of  TPSs. 

(1)  Identification  may  be  accomplished  by  assigning  part  numbers 
to  each  element  of  the  TPS  within  the  contractor’s  normal 
part  numbering  system. 

(2)  Control  and  updating  are  achieved  by  subjecting  all  elements 
of  TPS  to  formal  engineering  change  control. 

(3)  TPS  modifications  are  subject  to  management  control  and  are 
limited  to  modifications  to  correct  or  improve  the  TPS  or 
to  accommodate  a  UUT  change. 

(4)  Status  accounting  is  used  to  ascertain  the  identification  of 
all  UUT  product  configurations,  to  the  design  control  activity 
for  UUT  identification,  and  to  assure  that  a  TPS  has  been 
developed  for  each  legitimate  product  configuration  UUT. 

b.  Quality  Assurance.  Quality  Assurance  includes  the  generation  and 
use  of  «  quality  program  plan  which  assures  that  the  procured 
software  will  satisfy  the  support  requirements  of  the  prime  system. 

( 8) 

2 . 4  Digital  Test  Program  Generation  Systems  (DTPG) , 

This  guide  contains  the  analysis  of  29  viable  DTPG  systems.  By 
following  a  three  phase  selection  process  the  29  systems  are  reduced 
to  two  or  three  user  applicable  systems. 

The  first  phase  matches  user  test  programming  requirements  with  the 
features  and  capabilities  of  the  29  simulation  systems . 

The  second  phase  allows  selection  based  on  five  areas  of  technical 
performance : 

(1)  IC  modeling 

(2)  Circuit  modeling 

(3)  Good  circuit  simulation 

(4)  Fault  simulation 

(5)  Automatic  vector  generation 

The  third  phase  consists  of  the  user  exercising  the  remaining  2  or  3 
DTPG  systems  with  a  bread  board  card  representative  of  the  various 
types  of  circuitry  to  be  encountered.  Final  choice  considers  compari¬ 
son  of  the  test  results  plus  such  non-technical  aspects  as  warranty, 
maintenance  agreements  and  corporate  support. 
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Completion  Criteria. 

3.1  The  task  of  developing  the  initial  test  program  set  (TPS)  is  complete 
upon  delivery  of  an  acceptable  TPS  to  the  customer,  following  approved 
validation.  Monitoring  the  production  of  TPS  is  included  within  the 
requirements  of  Task  P1A.  The  task  of  logistics  planning  would  then 
have  the  added  advantage  of  knowing  the  probabilities  and  problems 
associated  with  TPS  ambiguity,  resulting  in  more  efficient  and  cost 
effective  maintenance. 


111-18 


APPENDIX  ''lCl 


This  appendix  consists  of  adapted  excerpts  which  treat  four  elements  of 
technique  applicable  to  test  program  set  (TPS)  development: 

•  TPS  cost  drivers 

•  Adapter  costs 

•  TPS  problems  and  solutions 

•  Automatic  Test  Program  Generation 

1.  Test  Program  Set  Cost  Drivers 

(9) 

a.  Testability 

The  largest  cost  driver  in  the  development  of  a  Test  Program  Set 
(TPS)  is  the  testability*  of  the  unit  to  be  tested.  Testability 
affects  the  analysis,  interface  adapter  design,  integration  and 
debug  cost  as  well  as  the  life  cycle  maintenance  costs  of  the  UUT. 

The  factors  of  testability  that  impact  TPS  costs  include  the 
following: 

•  Test  Point  Design  and  Placement 

•  UUT  Initialization 

•  UUT  Accessibility 

•  UUT  Packaging 

•  Adjustments  and  Select-at-Test  Components 

(9) 

b.  TPS  Isolation  Ambiguity  versus  UUT  Sparing 

The  logistics  sparing  for  UUTs  normally  takes  place  before  the  TPS 
is  designed,  and  is  based  on  statistical  failure  rates  of  components 
or  modules.  When  the  test  program  cannot  reach  unambiguous 
isolation  of  UUT  failures,  the  maintenance  technician  is  often 
faced  with  a  lack  of  spares.  Experimentation,  substitution,  and 
repair  by  stages  will  usually  result  in  the  UUT  being  restored  to 
service,  but  the  costs  involved  are  considerable.  The  test  station 
time  required  to  repair  such  a  failure  often  results  in  the  backup 
of  other  failed  UUTs  in  the  maintenance  pipeline  creating  a  main¬ 
tenance  backlog. 

It  is  suggested  that  sparing  be  accomplished  only  after  the  test 
program  analysis  is  completed.** 

*  "Testability"  here  refers  to  the  ease  and  simplicity  of  test  inherently 
contained  in  the  UUT.  -Ed. 

•♦Alternatively,  the  quantitative  spares  analysis  should  take  ambiguities 
into  account.  -Ed. 
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The  task  of  logistics  planning  would  then  have  the  added  advantage 
of  knowing  the  probabilities  and  problems  associated  with  TPS  am¬ 
biguity,  resulting  in  more  efficient  and  cost  effective  maintenance. 

C9) 

c.  Program  Structure 

One  of  the  reasons  for  high  TPS  maintenance  costs  is  the  total  lacn 
of  guidelines  and  specifications  regarding  TPS  stricture.  The 
classic  probl  m  with  all  software  maintenance  is  that  the  original 
programmer  is  not  available  for  the  correction  of  the  error,  and 
a  new  individual  n.ist  try  to  figure  out  what  was  meant  or  intended 
before  a  change  or  correction  is  attempted.* 

Well-defined  and  meaningful  entry  points  and  program  documentation 
can  help  reduce  field  maintenance  time,  and  also  reduce  the  t:me 
required  to  identify  and  correct  test  program  defects.  Entry7  point 
tables  that  identify  the  function  being  tested  reduce  repair  veri¬ 
fication  time  and  also  help  in  establishing  program  structure. 

(7) 

d.  Other  Significant  Cost  Contributors 

Other  significant  cost  contributors  not  readily  visible  in  the  TPS 
development  process  are: 

•  ATE  Maintenance 

•  UUT  Maintenance 

•  Original  Equipment  Manufacturer  (OEM)  consultation 

•  Autxnatic  Test  Program  Generator  (ATPG)  subcontracts 

•  Configuration  Management 

(10) ** 

2.  Multi-purpose  Adapters  for  Cost-reduction 

Assume  a  system  of  100  equipments  and  2000  modules  with  sets  of 
adapters  required  for  20  sices  at  $200  per  adapter.  The  recurring 
costs  of  adapters  can  vary  as  follows: 

o  If  each  mod  lie  requires  a  unique  adapter (2000  iapters  x  20  sites 
x  $200/adapter)  -  $8,000,000 

o  If  an  average  of  five  modules  are  served  by  each  unique  adapter 
(2000  t  5  adapters  x  20  sites  x  $200/a<’apter)  =  $1,600,000 

*  This  problem  is  common  to  all  programming,  yet  it  can  be  avoided  by 
simple  standardization  of  and  adequate  development  of  programming 
documentation.  -  Ed. 

**  This  excerpt  is  presented  to  illustrate  a  potential  tradeoff.  The  source 
article  did  not  treat  all  recurring  and  non-recurring  costs  impacted  by 
the  adapter  decision.  Numbers  are  quoted  directly  from  the  source. 
Intuitionally ,  the  more  complex  the  use  of  an  adapter,  the  higher  its 
cost  may  be.  Also,  there  are  impacts  on  the  non-recurring  engineering. 
-Ed. 
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•  If  each  equipment  is  served  by  a  unique  adapter  (100  equipments  x 
20  sites  x  $200/adapter)  =  $400,000 

The  analysis  needs  to  be  extended  to  consideration  of  non-recurring  as 
well  as  recurring  costs  and  must  also  consider  operating  and  support 
cost  impacts. 

3.  TPS  Problems  and  Solutions 

Inaccuracy  of  diagnosis  after  the  Test  Program  Sets  are  delivered  to 
the  user  has  been  troublesome,  (In  a  survey) : 

•  Undetected  non-UUT  (ATE  plus  interconnection)  r slated  failures 
caused  60%  of  the  mis-diagnosis. 

•  exit  of  scope  conditions  were  responsible  for  some  30%  of  diagnostic 
problems  (multiple -failures ,  chassis  wiring,  non-standard  failure 
modes ,  etc . ) 

•  Only  8%  of  the  erroneous  fault  isolation  could  be  traced  to  UUT 
oriented  TPS  software  errors. 

Conservative  estimates  indicate  that  the  customer  loses  the  equivalent 
of  more  them  30%  jf  the  original  procurement  cost  per  year  due  to  tur? 
type  of  mis-diagnosis.  The  cost  of  corrective  action  is  estimated  at 
less  them  one-tenth  this  amount  if  incorporated  before  TPS  production 
is  started  and  is  significantly  less  if  done  concurrently  with  TPS 
development . 

a.  Candidate  Corrective  Actions. 

(1)  Improve  ATE  real-time  self -test  capability. 

•  Utilize  local  microprocessor  monitoring  of  test  equipment 
functions . 

•  Implement  hardware  design  which  allows  optimum  failure 
monitoring  in  real-time. 

•  Provide  system  software  modules  to  allow  real-time  monitor¬ 
ing  during  testing. 

(2)  Improve  ATE  testability. 

•  Provide  necessary  hardware  to  allow  extensive  self-test  of 
the  ATE. 

•  Generate  ATE  resident  software  for  self-test. 

•  Allow  running  of  subsets  of  self-test  software  on  an  as- 
required  basis. 
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(3)  Include  in-process  testing  of  non-UUT  resources  prior  to  use. 

•  Provide  ATE  self-test  as  part  of  UUT  performance  testing. 

•  Provide  stimulus  and  measurement  limits  to  suit  specific 
UUT  tests. 

(4)  Include  in-process  calibration  to  attain  required  stimulus  and 

measurement  accuracies. 

•  Provide  measure-af ter-apply  techniques  for  stimuli. 

•  Provide  stimulus  standards  in  ATE  for  calibration  of 
measurement  devices. 

(5)  Provide  external  and  internal  hardware  to  achieve  near  100% 

ATE  confidence  at  the  JUT  connectors. 

•  Use  additional  hardware  in  interconnection  device. 

•  Apply  external  wraparound  techniques. 

(6)  Expand  data  collection  and  use  in  developing  meaningful  models 

for  ATE  effectiveness  acid  TPS  quality  to  include* 

•  Failure  data 

•  TPS  development  times 

•  On-station  activity. 

(12) 

4.  Automatic  Test  Program  Generation  (ATPG) . 

Digital  Automatic  Test  Program  Generation  is  defined  as  a  computer 
and/or  computer  program  which  aids  in  the  generation  of  test  programs 
for  digital  electronic  assemblies  in  an  automated  manner.  It  includes 
both  simulators  and  simulators  with  automatic  stimulus  generation  and 
is  used  for  both  the  development  and  maintenance  of  digital  test 
programs.  It  exists  because  the  manual  generation  of  digital  test 
programs  is  complex,  expensive,  skilled-labor-intensive  and  error 
prone.  The  figure  shows  the  general  ATPG  block  diagram.  The 
electronic  assembly  schematic  information  is  fed  into  the  system. 

The  ATPG  system  draws  on  an  IC  component  library  to  model  the 
schematic.  A  check  of  t'  e  model  is  accomplished.  The  stimulus  is 
then  generated,  either  manually  or  automatically.  The  ATPG  system 
then  simulates  the  electronic  assembly  and  determines  the  fault 
detection  and  fault  isolation  percentages.  The  patterns  are  then 
trams lated  into  the  specific  ATE  language  desired.  The  final  output 
is  a  Unit  Under  Test  (UUT)  test  program. 

*  Data  as  appropriate  should  be  provided  to  vendors  for  product  improvement . 
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The  generation  of  test  programs  is  proportional  in  complexity  to  the 
complexity  of  the  electronic  device  which  is  to  be  tested.  The  more 
complex  the  device,  the  more  difficult  it  is  to  generate  a  test 
program  for  that  device.  Testability  plays  a  key  role  in  the  genera¬ 
tion  of  test  programs.  Good  testability  can  greatly  ease  test 
program  generation  and  test  program  maintenance. 

A  choice  of  ATPG  systems  is  available  for  use.  The  following  criteria 
are  suggested  for  use  in  selection  of  a  system. 

•  Ability  to  Model  ICs  •  System  Cost  -  Acquisition  and  Use 

•  Stimulus  Generation  Capability  •  Vendor  Stability 

•  General  Ease  of  Use  •  Growth  Capability 

•  AJE  Compatibility  •  System  Maintenance  Availability 

•  System  Maturity 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F2A 

PHASE: 

FUNCTION: 

TASK  TITLE: 


TASK  OBJECTIVE: 
undetected  failures. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE _ INPUT  TO  T  OUTPUT  FROM  T 


e  System 

e  Partitioning,  System  Level 

e 

T  Analysis  Results 

Engineering 

BIT/ETE,  System  Functional 

(detail  design 

Diagram 

recommendat ions ) 

e  Design 

e  UUT  Test  Points,  HW  BIT, 

e 

T  Analysis  Results 

Engineering 

Inherent  T  Features 

(detail  design 

Schematics,  Logic  Diagrams 

recommendations ) 

e  Application 

e  SW  BIT,  Test  Program  Flow 

e 

T  Analysis  Results 

Software 

Charts  and  Program  Lists 

(detail  design 

- 

recommendations) 

e  Reliability 

e  FMEA,  System/Module 

e 

T  Analysis  Results 

Engineering 

Failure  Rates 

(detail  design 

recommendations) 

FULL  SCALE  DEVELOPMENT 

Analysis  of  Design  Test  Effectiveness 

Predict  the  levels  of  fault  detection  and  fault  isolation 
for  the  system,  subsystems  and  each  UUT. 

Achieve  a  detail  design  which  allows  for  optimal  detection 
and  isolation  of  failures  and  minimizes  the  occurrence  of 


COST  TRADEOFF  INTER-RELATIONSHIPS:  As  the  design  evolves,  consideration 

given  to  the  assurance  of  observability  and  controllability  may  add  to  design 
costs.  These  increases  will  be  offset  by  reduced  costs  in  test  program  set 
development,  by  reduced  costs  in  conducting  test  in  operations  and  by  reduced 
costs  in  logistics  (manpower,  spares,  and  test  equipment  usage). 


TASK  SYNOPSIS: 

1.  Task  Rt  Tuirements. 

1.1  A  formal  analysis  of  failure  modes  of  the  final  design  is  required  as  a 

followup  to  the  design  interfacing  conducted  in  Function  V2,  Incorporation 
of  Testability  into  System  Design  and  Function  V3,  Performance  of  Test 
Method  Tradeoffs  and  subsequent  coordination  efforts. 


Y 


A 
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The  task  is  initiated  in  conjunction  with  preparation  for  the  Critical 
Design  Review  and  continues  until  the  test  effectiveness  can  actually  be 
demonstrated  by  evaluation  of  developed  system  hardware. 

The  requirement  is  to  determine  that  expected  failures  can  be  detected 
and  isolated  so  that  mission  performance  effectiveness  is  achieved  and 
that  the  minimum  acceptable  number  of  faults  go  undetected.  Measuring 
failure  observability  and  test  controllability  is  a  primary  means  of 
assessing  testability  effectiveness  in  this  regard. 


2.  Implementation . 

The  total  task  consists  of  two  parts,  a  hardware  failure  analysis  to 
analyze  test  effectiveness  and  a  testability  analysis  model  to  analyze 
the  inherent  observability /controllability  of  the  configuration.  Maxi¬ 
mum  use  should  be  made  of  previous  analysis  tasks  (Function  V4)  performed 
during  the  validation  phase.  The  analyses  are  used  by  the  contractor  to 
support  pre-baseline  design  changes  and  to  provide  a  vehicle  for  govern¬ 
ment  review  at  both  system  and  subsystem  CDRs.  Portions  of  this  task  are 
closely  related  to  Test  Program  Set  development  of  Task  Pic. 

The  overall  concept  of  this  task  is  to  analyze  the  failure  modes  in  order 
to  obtain  measures  of  observability  and  controllability.  The  analyses 
should  be  computerized  as  feasible.  The  appendix  provides  some  primitives 
for  use  in  analysis. 


2.1  General  Notes  for  Detailed  Analysis. ^ 

The  failure  population  is  the  basis  for  test  derivation  (BIT  and/or 
external  test)  and  the  basis  for  test  effectiveness  evaluation.  With 
respect  to  the  failure  modes,  an  initial  step  is  to  determine  hardware 
partitions  for  failure  effects  analysis  considering  accuracy  required, 
cost  of  test  generation  and  simulation,  and  standardization  and  commonal¬ 
ity. 


a.  Testability  building  block.  The  lowest  level  of  hardware  partition 
may  be  referred  to  as  a  Testability  Building  Block  (TBB) .  A  TBB  may 
or  may  not  correspond  to  a  physical  partition  of  the  system  but 
typically  represents  a  LSI  component  or  a  small  printed  circuit  board. 
Component  structures  and  interconnections  may  also  be  fitted  into 
TBBs  such  that  the  relevant  failure  population  may  be  accurately 
modeled. 

b.  "Model’1  validation.*  The  structure  of  each  TBB  should  be  verified. 

A  simulation  technique  is  needed  to  apply  appropriate  portions  of  the 
functional  end-to-end  test  and  compare  simulated  responses  with  pre¬ 
dicted  responses  (or  with  responses  obtained  from  a  known  good 
hardware  unit,  in  subsequent  tasks) . 

*Subtasks  2.1b  and  2.1c  may  be  conducted  as  part  of  TPS  validation  at  such 
time  that  the  TPS  is  available. 
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c.  Test  stimulus  generation.*  This  derives  test  sequences  for  each  TBB 
using  the  most  cost-effective  methods.  Maximum  use  should  be  made  of 
functional  test  sequences  developed  for  end-to-end  testing.  Test 
algorithms  should  be  derived  so  as  to  facilitate  BIT  and  tester  imple¬ 
mentations  considering  software  looping  and  memory  constraints. 

<J.  Failure  response  data.  For  each  specified  failure  in  the  TBB,  the 
output  response  to  the  input  test  stimulus  should  be  determined  through 
a  simulation  technique.  Using  appropriate  failure  detection  criteria, 
a  record  should  be  kept  of  whether  or  not  the  failure  was  detected, 
and  if  detected,  of  the  expected  (good)  output  signature,  together 
with  the  predicted  output  signature  of  the  failed  unit. 

e.  Undetected  failures.  Undetected  failures  should  be  examined  to 
determine  if: 

(1)  The  failure  is  not  detectable  by  any  test  sequence.  Any  failures 
which  are  impossible  to  detect  due  to  redundancy  should  be  con¬ 
sidered  non-relevant  to  the  analysis. 

(2)  The  failure  is  potentially  detectable,  but  the  test  sequence  is 
deficient. 

(3)  The  failure  is  potentially  detectable,  but  the  unit's  hardware 
design  precludes  the  use  of  test  sequence  of  reasonable  length 
or  complexity. 

f.  False  alarm  rate.  All  GO  paths  of  BIT  software  and  Test  Program 
Sets  should  be  validated  using  a  known  good  system  or  UUT  prior  to 
test  design  approval.  The  correct  operation  of  concurrent  hardware 
BIT  and  application  software  error  processing  routines  should  be  veri¬ 
fied  by  analysis  of  the  design.  For  analog  systems,  or  digital  sys¬ 
tems  without  software,  all  GO  paths  of  each  BITE  indication  should  be 
validated  using  a  known  good  system  or  UUT  prior  to  final  hardware 
design  accaptance. 

g.  Failure  detection  times.  Failure  detection  time  (the  time  which 
elapses  between  the  occurrence  of  a  failure  and  the  detection  of  the 
failure  by  the  test  process)  should  be  expressed  as  two  or  more  time 
intervals  (representing  the  detection  latency  of  generic  test 
approaches,  e.g.,  concurrent  BIT,  periodic  BIT),  and  the  proportion  of 
failures  falling  within  each  interval.  For  example; 


Maximum  Detection  Time  %  Failures 

Less  than  1  second . 25 

Between  1  second  and  10  seconds . 65 

More  than  10  seconds  . 10 


h.  Failure  Isolation  times.  The  average  (or  maximum)  time  to  isolate 
failures  should  be  predicted  using  the  average  (or  maximum)  length  of 

*Subtasks  2.1b  and  2.1c  may  be  conducted  as  part  of  TPS  validation  at  such 
time  that  the  TPS  is  available. 


the  diagnostic  test  sequence  plus  an  estimation  of  time  for  any  manual 
intervention  required.  Times  should  be  predicted  for  both  BIT  and  ATE 


i.  Government  furnished  equipment  data.  Testability  parameter  values 
should  be  requested  from  the  procuring  agency  and  used  in  the  test¬ 
ability  prediction.  If  the  values  are  unavailable  or  unknown,  estimate 
the  values.  If  the  estimated  or  furnished  values  are  incompatible 
with  the  intended  use,  or  analysis  indicates  that  the  system/equipment 
will  not  satisfy  the  operational  or  maintainability  requirements  based 
upon  these  values,  these  problem  areas  should  be  identified  and  the 
procuring  activity  advised  with  proposed  alternative  courses  of  action. 

j.  Corrective  action.  This  may  include  additional  patterns  in  the  test 
sequence,  where  feasible,  to  meet  failure  coverage  and  isolation 
requirements.  If  additions  to  the  test  sequence  are  not  possible  or 
cure  not  reasonable,  the  T  engineer  can  propose  appropriate  design 
changes  for  the  hardware  to  improve  its  controllability/observability 
characteristics.  The  proposed  modification  should  be  modeled  and 
simulated  using  modified  test  sequences  to  determine  if  the  changes 
improve  the  failure  coverage  and  isolation  sufficiently. 


2.2  Observability/Controllability  Analysis. ^ 

A  Testability  Analysis  for  each  Configuration  Item  (i.e. ,  potential  UUT) 
should  be  developed  and  maintained.  Maximum  use  should  be  made  of 
analysis  developed  during  preliminary  design  to  determine  inherent  observ¬ 
ability  and  controllability.  The  overall  testing  structure  should  be 
represented  by  a  hierarchy  of  analysis  representing  various  levels  of 
testing.  Higher  levels  in  the  hierarchy  should  be  defined  in  terms  of 
the  lower  level  blocks  and  the  stimulus/response  requirements  for  each 
lower  level  block.  An  analysis  may  correspond  to  any  physical  or  func¬ 
tional  grouping.  Certain  analyses  represent  the  testing  structure  for 
UUTs  and  .iius  support  TPS  development.  Each  level  of  Testability  Analysis 
should  permit  the  following  determinations: 

a.  Identify  the  response  observation  points  (for  BIT,  ATE,  and  manual 
tfcuit  for  this  level. 

b.  Identify  the  test  modes  and  stimulus  injection  points  for  this  level. 

c.  Verify  that  signal  paths  and  control  paths  exist  to  provide  stimulus 
to  lower  level  blocks  as  determined  by  their  test  requirements  and 
identify  the  stimulus  required. 

d.  Verify  that  signal  paths  and  control  paths  exist  to  propagate  the 
resulting  test  response  of  lower  level  blocks  to  observation  points 
at  this  level  and  identify  the  resulting  failure  responses. 
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2.3  Military  Documents. 

The  following  documents  Lay  aid  in  task  performance: 

a.  Failure  Population  Definition,  MIL- STD-471,  Maintainability  Verifica- 
tion/Demonstrati on/Evaluation 

b.  Failure  Mode  and  Effects  Analysis,  MIL-STD-2070(AS) ,  Procedures  for 
Performing  a  Failure  Model,  Effects  and  Criticality  Analysis  for 
Aeronautical  Equipment 


3.  Completion  Criteria. 

This  task  should  be  merged  with  the  development  of  Test  Program  Sets  and 
the  Testability  Demonstration .  It  is  complete  when  the  Testability 
Demonstration  results  have  been  approved  by  the  government. 
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APPENDIX  F2A1 


This  Appendix  presents  details  of  two  measures  which  may  aid  in  a  determina¬ 
tion  of  effectiveness: (1)  Weighted  Failure  Coverage,  and (2)  Failure  Resolution^1) 

1.  Failure  Coverage. 

The  unweighted  failure  coverage  is  defined  as  K/S  where  K  is  the  number 
of  failures  detected  and  S  is  the  total  number  of  failures  in  the  failure 
population,  corrected  for  impossible  detects.  If  the  analysis  data  base 
includes  failure  rate  data,  the  weighted  failure  coverage  is  calculated 
as  the  sum  of  the  failure  rates  of  the  detected  failures  divided  by  the 
sum  of  the  failure  rates  of  all  the  specified  failures.  Individual 
failures  within  a  component  should  be  assigned  an  equal  proportion  of  the 
component ' s  total  failure  rate  unless  more  accurate  failure  data  are 
available  and  feasible  to  apply. 


Failure  Resolution. 

The  degree  of  failure  isolation  is  calculated  using  the  following 

methodology. 

a.  Failure  signature  data.  Data  are  required  which  correlates  each 
detected  failure  with  the  signature  it  produces  during  testing.  The 
data  are  most  conveniently  ordered  by  signature  and  by  failing  module 
within  each  signature  (fault  dictionary  format) . 

b.  Substitution  method.  The  failure  resolution  calculations  depend 
upon  the  substitution  method  used  to  effect  repairs.  If  all  modules 
under  a  signature  are  replaced  as  a  block,  equation  (3a)  is  used.  If 
one  module  of  the  signature  group  is  replaced  and  the  test  rerun  for 
PASS/FAIL,  equation  (3b)  is  used  (both  equations  are  stated  under 
Paragraph  2d,  Calculation,  below). 


c.  Notation. 


number  of  unique  signatures  in  dictionary 
signature  index 

number  of  modules  listed  in  signature  i 
module  index  within  signature 

number  of  faults  in  module  j  which  produce  signature  i 
failure  index  within  module 

failure  rate  for  kth  failure  within  jth  module  within  ith 
signature  (see  note) 
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xu 


2 

1Cf»1 


ijk 


failure  rate  for  jth  module  for  failures 
providing  signature  i  (see  note}* 


M. 


2 

j-i 


'ij 


failure  rate  for  failures  producing  signature  i 


-  2  *i 


i=l 


overall  failure  rate 


Mmax  =  maximum  (M^)  =  worst  case  isolation 

NOTE:  If  detailed  failure  data  is  unknown,  apportion  module  faults 
as: 


x  =  i _ lil 

i]  \ Total  Module  Faults , 


module 


d.  Calculations. 


A.  -  %  signatures  which  have  an  ambiguity  of  ^modules 

*  ..  r.  .  « 


i 

N 


(Xt)  x  100 


where  Xi 


£*Tl 


1  if  M± 

0  if  M±  -  l 


(1) 


%  signatures  with  ambiguity  '<L 
L 


l 


where  one  or  more  values  of  L  are  defined  in 
the  specification. 


(2) 


RRi  =  replacement  rate  for  signature  i  based  upon  failure  rates 
"Replace  all"  strategy: 


“i  ■  xi 


(3a) 


"Replace  one  at  a  time  and  retest"  strategy: 
1  Mi 

**1  "  —  V  j*ij 


(3b) 


M. 
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**1  X  100 
\ 


(4) 


PS.  *  %  replacements  due  to  signature  i  * 


PR. 


%  replacements  due  to  signatures  containing 


PR’  - 

u 


i  modules  •  \  XiPSi 

i«l 

%  replacements  with  ambiguity 


PR. 


1  -1 
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(6) 
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PHASE: 


FUNCTION-. 


TASK  TITLE: 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F3A 

FULL  SCALE  DEVELOPMENT 

Testability  Cost/Benefits  Analysis 

Develop  Testability  Benefits  and  Costs 


TASK  OBJECTIVE:  Provide  visibility  for  non-recurring  and  recurring  costs 

penalties  and  offsetting  benefits  and  savings  due  to  in¬ 
corporation  and  exploitation  of  testability  in  the  prime  system. 

DESIGN  DISCIPLINE  TESTIBILITY  JT)  INTER-RELATIONSHIPS : 


T  INTERFACE 


e  Application 
Software 


e  Design 

Engineering 

e  Logistic 
Support 

e  Maintainability 
Engineering 

e  Maintenance 
Engineering 

e  Support  and 
Test  Equipment 

e  Manufacturing 
Test  Engineering 


INPUT  TO  T 


e  Differences  in  Number  of 
Instructions,  Memory  Size, 
Documentation 

e  Differences  in  Development 
(Design)  Josts,  Number 
Circuits,  Documentation. 

•  Differences  in  Spares, 
Repair  Turnaround  Times. 

e  Differences  in  MTTR 


OUTPUT  FROM  T 


e  Guidance  to  all  other 
disciplines  to  assist  in 
developing  valid  informa¬ 
tion  for  Cost/Benefits 
Analysis .  Guidance  in- 
includes  description  of 
baselines  for  comparisons 
and  differences  associated 
with  Testability. 


e  Training 


e  Maintenance 
Documentation 

e  Reliability 

e  Life  Cycle 
Costing 


e  Differences  in  Repair  Levels, 
Facilities,  Skill  Levels 

e  Differences  in  Test  Equipment 
Complexity,  Cost,  Maintenance 

e  Differences  in  Fault  Detection 
Time,  Fault  Isolation  Time, 
Alignment/Calibration  Time, 
Checkout/Selloff  Time 

e  Differences  in.  Personnel 
Training  Time,  Con?) laxity  of 
Training  Preparation, 

Materials 

e  Differences  in  Manual  Pre¬ 
paration,  Complexity 

a  Differences  in  Failure  Rates 


Differences  in  LCC  &  LCC 
elements;  Guidance  in  Use 
of  Current  LCC  Techniques 


a  Documents  both  tangible  & 
intangible  Testability 
Impacts  &  provide  inputs 
to  LCC  analysis. 
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COST  TRADEOFF  INTER- RELATIl  .ISHIPS:  The  task  itself  treats  cost 

differences  extensively  although  it  represents  a  recapitulation  rather  than 
a  cost-driving  effort. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

The  essential  requirement  is  to  achieve  a  presents t  '  of  benefit  vs. 

penalties  which  is  clear,  logical  and  auditable.  In  presenting  costs, 
the  differences  (between  design  approaches  reflecting  varying  degrees 
of  testability)  need  to  be  emphasized  to  a  greater  degree  than  the 
ah solute  costs. 


2.  Implementation . 

The  Design  Discipline  Interrelationship  chart  at  the  heading  of  this 
task  compendium  is  indicative  of  the  implementation.  The  LCC  analyst 
is  probably  the  most  valuable  of  the  contacts  for  this  task,  inasmuch 
as  the  techniques  to  be  applied  cure  primarily  those  of  LCC  analysis. 

The  techniques  include  those  of  various  repair  level  analysis  methods, 
(e.g. ,  AFSC  800-4,  MIL-STD-1390)  as  well  as  USAF  Logistic  Support  Cost 
and  LCC  models.  Beyond  these  techniques,  each  testability  analyst  will 
need  to  tailor  methodology  and  analysis  for  application  to  each  particu¬ 
lar  program.  One  literature  extract  is  quoted  which  provides  some 
detailed  guidance  in  analysis  of  Test  Program  Set  costs  and  benefits. 


2.1  Costs  Analysis. 

Costs  fall  primarily  into  development  and  recurring  areas. 


a.  Development  costs.  The  costs  of  a  development  program  which 
are  attributed  to  testability  design  are  visible  and  identifiable 
through  the  assignment  of  appropriate  work  elements  in  the  Work 
Breakdown  Structure.  Costs  for  testability  design,  analysis, 
and  data  preparation  during  a  formal  testability  program  should 
be  included. 


b.  Recurring  costs  and  penalties.  Identify  those  costs  and 
operational  penalties  associated  with  the  incorporation  of 
testability  into  the  system  or  equipment.  Itemized  testability 
costs  should  include,  but  may  not  be  limited  to,  the  followings 
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•  Per  unit  cost  of  additional  hardware  required  for  BIT  and 
testability  capabilities 

e  Volume  and  weight  required  for  additional  hardware ,  additional 
connectors,  and  increased  modularity 

e  Power  required  for  additional  hardware 

e  Computer  memory  required  for  test  programs 

e  Possibility  of  system  interruption  due  to  failure  within 
BIT  circuitry 

e  Reliability  impact  due  to  additional  hardware 


Test  Program  Set  Development  Costs.  3  The  development  of  test 
program  sets  (TPS)  for  the  prime  equipment  is  an  initial  nonrecurring 
cost.  TPS  development  costs  must  be  considered  for  both  ATE  and  MTE. 
It  is  important  to  retain  visibility  of  costs  incurred  independently 
of  a  concerted  testability  program;  most  procurements  require  many 
of  these  elements  even  without  mention  of  optimized  testability. 

Test  Program  Set  Development  includes  the  following  elements  of  cost: 


Element 


Labor 


Material 


(1)  Acquire  basic  data  on 
TJUT  for  test  analysis 


In-house  development  Outside  purchase 
or  detailed  review  from  vendor 
of  vendor  data 


(2)  Develop  test  strategy 
and  document  in  the 
form  of  Diagnostic 
Flow  Charts  (DFC)  and 
test  setup  diagrams 
compatible  with 
tester  to  be  used. 


Perform  test  require¬ 
ment  analysis  (TRA) . 
Prepare  test  require¬ 
ment  document  (TRD) , 
including  test 
interconnect 
diagram. 


Digital  stimulus/ 
response  data  may 
be  purchased  from 
vendor  or  produced 
using  automatic 
test  program 
generation  tech¬ 
niques. 


(3)  Interface  hardware 
design  and  model 


Document  interface 
device  (ID)  hard¬ 
ware  and  build 
development  model. 


Hardware  costs  for 
material  s  elec¬ 
trical  components. 


(4)  Test  Program 

Instructions  (TPI) 


Develop  step-by-  Artwork  costs  & 

step  procedure  printing 

for  on-station 
testing  of  CUT  in¬ 
cluding  operator 
actions  to  correct 
malfunctions  de¬ 
tected. 
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Element 


Labor 


Material 


Code/Compile 
(ATE)  or  Test 
Procedure  (MTE) 

Generate  test  pro¬ 
gram  software  using 
tester's  higher  order 
language  (ATE)  or  pre¬ 
pare  detailed  test 
procedure  (MTE) . 

Compiler  opera¬ 
tions  and 
maintenance  (ATE) 

TPS  Integration 

^  Verification  of  test 
programs  integrity 
by  actual  demon¬ 
stration  on  station 

Repair  facility 
support  and 
tester  maintenance 

a)  UUT  test  perfor¬ 
mance  debug 

b)  UUT  test  diag¬ 
nostic  debug 

c)  Fault  simulation 

d)  Test  Program,  ID, 
and  TPI  Updates 

Formal  Sell-Off 
(Validation) 

Demonstration  for 
inspection  personnel 
and  customer  of  TPS 
integrity.  Includes 
functional  testing  & 
sample  fault  insertion. 

Same  as  (4)  ,  (5) 
and  (6) 

Verification 

Demonstration  of  first 
production  article  on 
tester  and  fleet  intro- 
ducti on . 

Field  service 
expenses 

The  complexity  of  TPS  design  will  cause  a  wide  variance  in  total  manhours 
due  to  degree  of  testing  required  for  the  following  reasons: 


(1) 

Level  of  Repair 

Functional  test  with  no  diagnostics  to 
full  diagnostics. 

(2> 

Tester  compatibility 

Complexity  of  ID  design  to  mate  tester 
to  UUT. 

(3) 

Test  ambiguity 

Degree  of  fault  isolation  to  groups  of 
SRUs,  single  SRUs  or  group  of  components 

(4) 

Software  update 
capability  (ATE) 

The  flexibility  of  ATE  software  update 
from  on -station  patching  (simple)  to 
batch  compilation  on  a  separate  computer 

(complex) . 
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Degree  of  sampling  required  to  sell  off 
TPS. 


(5)  Verification  and 
Validation  (V&V) 
Requirements. 


2.2  Fundamental  Benefit  Areas. 

Cost  elements  treated  in  current  LCC  techniques  should  be  screened  for 
applicability  to  this  task.  Penalties  and  benefits  may  be  expressible  in 
financial  terms  but  will  generate  from  analysis  of  performance,  opera* 
tional  and  support  inpacts .  The  following  excerpts ( D  provide  a  point 
of  departure  for  defining  detail  elements  to  be  treated. 

a.  Development  benefits.  Estimate  the  cost  reductions  (due  to  the 
testability  program)  ins 

e  Test  Program  Set  (including  interface  device)  size,  complexity, 
development  time  and  cost. 

e  Diagnostic  software  size,  complexity,  development  time  and  cost. 

e  Automatic  test  generation  software  complexity,  effectiveness  and 
cost. 

e  Factory  test  time  and  cost  for  all  levels  of  assembly, 
e  TPS/diagnostic  software  costs  for  later  hardware  modifications. 

b.  Operational  and  support  benefits.  Estimate  the  cost  benefits  of 
the  testability  program  on  system  readiness  in  terms  of  fewer  systems 
assets  needed  to  meet  operational  readiness.  Support  costs  are  sub¬ 
stantially  reduced  particularly  in  the  following  elements! 

e  Technical  Manual  acquisition 
e  Initial  and  replacement  training 
e  Initial  and  replenishment  spares 
e  Skill  levels 
e  Test  and  repair  manhours 
e  Test  equipment  maintenance 
e  Transportation  of  repair  items 

2.3  Documentat ion . 

The  documentation  of  the  results  of  the  cost/benefits  analysis  should 
show  that  inputs  from  all  testability  interfaces  are  complete,  and  valid 
and  all  testability  affected  items  are  accounted  for. 

3.  Completion  Criteria. 

The  task  is  completed  upon  government  acceptance  and  approval  of  the 
documented  results. 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F4A 


PHASE: 


FUNCTION: 
TASK  TITLE: 


FULL  SCALE  DEVELOPMENT 
CDR  Support 

Review  testability  features  and  predicted  testability 
parameters  for  each  Cl 


TASK  OBJECTIVE:  Assure  that  testability  features  and  predicted  testability 

parameters  are  included  as  part  of  the  design  approved  for 
development  into  the  production  configuration. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACES _ INPUT  TO  T  _ OUTPUT  FROM  T 


•  All  Design 
Activities 


e  Consultation  and 
Coordination 


e  Consultation  and 
Coordination 


e  Program 
Management 


e  Critique  of  CDR  Review 
Material 


e  CDR  Review  Material, 
Presentation  to 
Contractor/Go  vemment 
Program  Management 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Not  relevant  to  this  task. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

The  task  requirement  is  to  present  at  the  CDR,  the  testability  aspects  of 
the  detailed  engineering  design  so  that  they  are  incorporated  as  require¬ 
ments  to  be  met  during  evolution  of  the  design  into  manufacturing  con¬ 
figuration.  Proper  testability  representation  in  the  CDR  process 
obviates  t^p  need  for  subsequent  complex  and  expensive  redesign  to 
accommodate  testability. 


2.  Implementation . 

Extensive  preparation  in  terms  of  liaison  with  other  disciplines  is 
advisable  so  that  the  Testability  position  presented  at  CDR  will  be  non- 
controversial.  CDR  should  be  followed  up  by  monitoring  actions  assigned 
which  are  relevant  to  testability. 
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2 . 1  Preparation . 


Preparation  for  the  CDR  consists  of  review  of  all  generated  testability 
materials  and  review  of  the  development  specifications,  followed  by 
organization  of  the  major  testability  elements  for  concise,  unambiguous 
presentation.  Coordination  should  be  conducted  as  necessary  to  assure 
validity  of  data  and  understanding  by  the  other  disciplines  of  the  test¬ 
ability  viewpoint. 

The  source  documents  and  data  include  validation  phase  testability 
analysis  report,  maintainability  documentation,  detailed  engineering 
drawings  of  each  Cl,  UUT,  Test  Equipment,  diagnostic  software  design, 
flow  diagrams,  and  backup  material. 

2.2  Cl  Testability  Dataf ^ 

Testability  data  for  each  Cl  is  reviewed,  including  built-in  test  methods 
(hardware,  firmware  and  software)  and  predictions  on  failure  coverage, 
failure  isolation  levels  and  times,  and  false  alarm  rates.  The  achieve¬ 
ment  of  qualitative  testability  requirements  is  verified  as  part  of  the 
Functional  Configuration  Audit.  The  detailed  engineering  drawings  of  the 
product  specification  are  reviewed  and  prototype  hardware  is  inspected  to 
insure  compliance  with  test  requirements  in  the  development  specification, 
including : 

a.  Mechanical  and  electrical  compatibility  between  the  prime  equipment 
and  on-line  readiness  monitoring  equipment  for  all  parameters 
specified  to  be  monitored. 

b.  Modular  construction  of  the  equipment  to  support  testing  and 
maintenance  requirements. 

2.3  JUT  Testability  Data. ^ 

Testability  data  for  each  module  of  the  Cl  is  reviewed,  including  test 
interfaces,  test  control,  and  test  access  for  both  ETE  and  BIT  environ¬ 
ments.  Predictions  are  made  on  the  levels  of  failure  coverage  and  failure 
isolation  (within  the  module)  that  may  be  achieved.  The  detailed  engineer¬ 
ing  drawings  of  the  product  specification  are  reviewed  and  the  prototype 
hardware  is  inspected  to  insure  compliance  with  test  requirements  in  the 
development  specifications  including  the  mechanical  and  electrical  com¬ 
patibility  between  the  specified  ETE  and  each  prime  equipment  module 
designated  to  be  a  unit  under  test  for  that  ETE. 

2.4  Test  Consistency. ^ 

The  capabilities  of  built-in-test,  external  test,  and  the  corresponding 
maintenance  documentation  for  each  are  reviewed  for  consistency  with  the 
maintenance  concept  and  Level  of  Repair  Analysis,  plans  for  orderly 
progression  from  organizational  level  testing  to  intermediate/depot  level 
testing  are  reviewed.  The  contractor  will  plan  for  maximum  utilisation  of 
designated  ETE  during  production  and  test  and  evaluation. 
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2.1  Configuration  Control. ^ 

The  configuration  control  to  be  exercised  in  accordance  with  MIL-STD-480 
over  each  Cl,  diagnostic  software  CPCI ,  ETE,  and  Test  Program  Set  is 
reviewed. 


3.  Completion  Criteria. 

3.1  Government  Approval. 

Government  approval  of  the  equipment/development  specifications,  together 
with  approval  of  CDR  minutes  comprise  completion  of  the  task.  Actions 
arising  from  the  CDR  should  be  assigned  as  management  attention  items. 


111-33 


TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F5A 


PHASE: 


FULL  SCALE  DEVELOPMENT 


FUNCTION: 


TASK  TITLE: 

TASK  OBJECTIVE: 


Testability  Demonstration  Plan  Preparation 

Prepare  Testability  Demonstration  Plan 

Document  for  government  approval  the  descriptions  of  the 
procedures  for  conducting  the  Testability  Demonstration. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  Design 

e 

HW  BIT  Features,  Equipment 

e 

T  Demonstration  Plan 

Engineering 

T  Features  Designed  In 

Requirements 

e  Application 

e 

SW  BIT  Features,  SW  Diag¬ 

e 

T  Demonstration  Plan 

Software 

nostic  Tests 

Requirements 

e  Support 

e 

ATE /Manual  Test  Equipment 

e 

T  Demonstration  Plan 

Equipment 

Availability 

Requirements 

e  Test 

e 

Test  Program  Features 

e 

T  Demonstration  Plan 

Engineering 

Requirements 

e  Program  Manager 

e 

Test  Schedule  &  Review  Dates 

e 

T  Demonstration  Plan 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Not  applicable  to  this  task. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

This  task  requires  the  preparation  of  a  plan,  formatted  in  accordance 
with  the  CDRL  for  the  conduct  of  a  testability  demonstration.  The 
demonstration  may  also  be  governed  to  some  extent  by  the  quality  assur¬ 
ance  procedures  of  the  contract.  The  demonstration  is  to  be  performed 
to  verify  the  achievement  of  specified  testability  parameters  on  major 
end  items,  similarly  to  the  demonstrations  of  achievement  of  maintain¬ 
ability  parameters  conducted  in  accordance  with  MIL-STD-471. 

This  task  consists  of  development  of  the  plan  by  which  the  actual  demon¬ 
stration  will  be  conducted.  Task  Reference  Number  FSB  provides  for 
actual  conduct  of  the  testability  demonstration. 


2.  Implementation . 

This  task  involves  preparation  of  the  testability  demonstration  plan. 

A  testability  demonstration  evaluates  the  effectiveness  of  on-line  tests 
and  off-line  test  interfaces. 

The  techniques  Which  follow  here  address  the  concept  of  testability 
demonstration,  the  testability  task  pool  and  demonstration  procedures 
as  background  to  under standing  plan  formulation.  A  candidate  DID  from 
the  literature  is  also  shown. 


2.1  Testability  Demonstration. 

Demonstration  tests  are  performed  on  major  end  items,  including  complete 
systems,  using  the  anticipated  test  environment  (test  equipment,  test 
software,  and  test  documentation) .  The  testability  demonstration  should 
be  accomplished  during  Full  Scale  Development  qualification  testing 
utilizing  contractor  personnel  and  contractor  facilities,  and  should  be 
monitored  by  the  procuring  activity. (U 


Demonstration  procedures  include  the  introduction  of  actual  failures 
into  equipment  for  the  verification  of  BIT,  test  software,  and  main¬ 
tenance  error  dictionaries.  This  demonstration  is  in  addition  to  any 
specified  Maintainability  Demonstrations  which  address  maintenance  . 
procedures,  equipment  accessibility,  technician  skill  levels,  etc., 
as  well  as  fault  detection  and  isolation  considerations. (14) 


a.  Testability  Task  Pool  Concept.  The  demonstration  of  testability 
parameters  should  be  accomplished  through  the  completion  of  several 
testing  tasks.  These  tasks  are  selected  at  random  from  a  task  pool 
defined  by  the  contractor  prior  to  the  demonstration.  Each  task 
defines  one  failure,  the  method  of  inserting  or  simulating  the 
failure,  the  expected  failure  response,  and  the  detailed  operating 
procedures  for  obtaining  the  initial  failure  detection  indication. 
Failures  may  be  simulated  by  introduction  of  faulty  parts,  deliberate 
misalignment,  open  leads,  shorted  parts,  etc.  The  task  pool  should 
contain  at  least  twice  the  maximum  number  of  tasks  required  for 
demonstration  and  should  be  specified  in  the  Testability  Demonstra¬ 
tion  Plan.  The  task  pool  should  be  prepared  by  the  contractor  in 
accordance  with  the  stratification  procedure  described  in  Appendix  A 
of  MIL-STD-471  •  to  insure  that  a  proportionately  representative 
sample  of  failure  modes,  BIT  design  features,  and  ATE  design  features 
are  selected  to  be  exercised.  The  number  of  tasks  to  be  demonstrated 
should  be  determined  using  MIL-STD-105  methods  based  upon  the  size  of 
the  failure  population  and  the  desired  test  level. 
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2.2  Testability  Demonstration  Procedures. H.) 

The  plan  to  conduct  the  testability  demonstration  can  be  developed  in 
accordance  with  the  following  paragraphs  : 

a.  Item-related  information.  The  plan  should  contain  a  list  of  all 

items  to  be  demonstrated ,  and  should  contain  for  each  item: 

(1)  The  identification  of  quantitative  testability  requirements 
to  be  demonstrated. 

(2)  A  list  of  documentation  required  prior  to  the  start  of  the 
demonstration . 

(3)  The  mechanism  for  inserting  failures  into  the  item. 

(4)  The  procedure  for  applying  stimulus. 

(5)  The  procedure  for  measuring  and  observing  the  test 
results . 


b.  General  information.  The  plan  should  contain  the  following  general 

information: 

(1)  The  identification  of  the  demonstration  site,  contractor 
organization  and  responsibilities,  and  government  responsi¬ 
bilities. 

(2)  The  membership  and  duties  of  the  test  team. 

(3)  The  methodology  used  in  defining  the  failure  population 
for  the  task  pool. 

(4)  The  size  of  the  task  pool. 

(5)  The  procedure  for  selecting  failures  from  the  task  pool 
for  insertion  into  the  items. 

(6)  A  listing  of  all  support  items  required  to  calibrate 
equipment,  insert  failures,  apply  tests  and  observe 
responses. 

(7)  A  methodology  for  the  interpretation  and  analysis  of 
results  (i.e.,  fault  detected  with  isolation,  fault 
detected  without  isolation,  fault  not  detected,  etc.) 

(8)  The  establishment  of  acceptance  criteria  for  the  demonstra¬ 
tion  (fault  coverage,  degree  of  fault  isolation,  MTL-STD-471 
test  method  to  be  used) . 
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(9)  A  plan  for  taking  corrective  action  as  may  be  needed. 

(10)  The  schedule  for  demonstration,  including  identification  of 
any  other  testing  being  performed  concurrently  and  its  inter¬ 
face  with  this  test. 

c.  False  Alarm  Sate  Demonstration  Information. 

(1)  False  alarm  test  information.  For  the  conduct  of  the 
false  alarm  rate  demonstration  as  required  by  the  contract, 
the  plan  should  identify  the  tests  to  be  used  as  data 
sources,  and  the  MIL-STD-781  Test  Flan  to  be  used. 

(2)  False  alarm  rate  demonstration.  Plan  to  demonstrate  the 
achievement  of  the  specified  false  alarm  rate  as  required 
by  the  quality  provisions  of  the  contract.  The  false  alarm 
rate,  expressed  in  terms  of  average  number  of  false  alarms/ 
hour  of  equipment  or  system  operating  time,  should  be 
demonstrated  from  false  alarm  data  resulting  from  controlled 
tests  (e.g. ,  reliability  demonstration  tests,  performance 
tests) .  The  contractor  and  the  procuring  activity  should 
jointly  determine  the  specific  data  sources  to  be  utilized, 
and  the  contractor  should  prepare  a  sub-plan  as  part  of  the 
Testability  Demonstration  Plan  defining  the  procedures  to 

be  utilized  in  the  collection  and  documentation  of  such  data. 
Allowance  may  be  made  to  continue  to  collect  and  record  false 
alarm  data  through  operational  testing  if  necessary  to 
identify  design  deficiencies. 

(3)  Demonstration  criteria.  The  False  Alarm  Rate  Demonstration 
should  be  based  upon  the  criteria  of  MIL-STD-781,  Reliability 
Tests.  The  MIL-STD-781  test  plan  to  be  used  should  be  based 
upon  the  time  available  for  test,  acceptable  decision  risks, 
and  specified  discrimination  ratio.  The  cumulative  period 
of  operating  time  must,  as  a  minimum,  include  the  operating 
time  duration  of  the  reliability  demonstration  test(s). 

Each  confirmed  false  alarm  should  be  treated  as  a  relevant 
failure.  The  specified  false  alarm  rate  should  be  input  to 
the  test  plan  as  a  component  of  the  total  failure  rate.  The 
equipment  should  be  considered  acceptable  with  respect  to 
false  alarm  rate  if  the  acceptance  criteria  of  the  selected 
MIL-STD-781  test  plan  is  met. 

(4)  Related  false  alarms.  If  two  or  more  observed  false  alarms 
are  positively  attributable  to  a  single  design  problem  which 
has  been  identified  for  correction,  only  one  false  alarm  is 
chargeable  to  the  false  alarm  count. 
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d.  Corrective  Action.  Planning  for  failure  to  pass  the  testability 
demonstration  should  require  identification  of  appropriate  redesign 
efforts  by  the  contractor.  Provide  time  in  the  development  schedule 
to  allow  for  correction  of  deficiencies  and  repeating  of  failed 
tests.  Redesign  considerations  should  include: 

(1)  Redesign  of  prime  equipment. 

(2)  Redesign  of  test  equipment. 

(3)  Redesign  of  interface  devices. 

(4)  Redesign  of  built-in-test  circuits. 

(5)  Redesign  of  diagnostic  software. 

(6)  Redesign  of  ATE  software. 

(7)  Correction  of  maintenance  documentation. 

(8)  Correction  of  models  used  for  testability  analysis. 


e.  Other  test  data.  In  addition  to  formal  demonstration,  plan  to 
maintain  a  record  of  all  failures  of  assembled  equipment  during  all 
testing  throughout  the  contract.  At  each  occurrence  of  failure  the 
built- in-test  function  should  be  exercised,  if  applicable,  and  an 
entry  on  the  failure  report  made  to  indicate  compliance  or  non- 
compliance  with  the  requirements  for  failure  detection  and  isolation. 
A  description  of  the  malfunction,  including  failure  symptoms,  should 
be  provided  and  documented  in  the  Testability  Analysis  Report. 

(141 

2.3  Testability  Demonstration  Plan  Data  Item  Description. 

A  Data  Item  Description  (DID)  has  been  written  and  proposed  as  the 
standard  for  documenting  the  testability  demonstration  plan.  The 
following  excerpt  from  that  DID  supplements  the  foregoing  discussion  of 
the  contents  of  the  plan. 

a.  Description.  This  plan  describes  the  contractor's  procedures 
for  conducting  the  testability  demonstration  and  is  used  by  the 
procuring  activity  to  evaluate  the  procedures.  The  Testability 
Demonstration  may  be  used  to  evaluate  the  effectiveness  of  built-in 
hardware,  diagnostic  software,  and/or  design  for  testability  features. 

b.  Application.  This  Data  Item  Description  is  applicable  to  develop¬ 
ment  contracts  during  the  full  scale  development  phase  which  require 
the  formal  demonstration  of  testability  requirements  in  systems  and/ 
or  Configuration  Items.  This  plan  is  not  required  if  the  testability 
demonstration  is  an  integral  part  of  the  overall  Qualification 
Testing  and  is  adequately  documented  in  the  Qualification  Test  Plan. 
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c.  Preparation  Instructions.  The  Testability  Demonstration  Plan  should 
contain  the  plans  and  procedures  for  conducting  a  demonstration  of  the 
effectiveness  of  built-in-test  and  design  for  testability  features, 
and  the  validity  of  :he  Testability  Analysis  Model  as  required  in 
MIL- STD- XXX. 

The  Plan  should  contain  a  list  of  all  items  to  be  tested,  and  should 
contain  as  a  minimum  for  each  item: 

(1)  A  listing  of  qualitative  and  quantitative  testability 
requirements. 

(2)  The  identification  of  those  qualitative  and  quantitative 
testability  requirements  to  be  demonstrated. 

(3)  The  methodology  for  demonstrating  the  validity  of  the  models 
used  for  testability  analysis. 

(4)  A  list  of  documentation  required  prior  to  the  start  of  the 
demonstration  (stimulus/response  data,  response/fault 
correlation  data,  testability  analysis  models,  diagnostic 
software  design  documentation,  etc.). 

(5)  -The  identification  of  the  demonstration  site,  contractor 

organization  and  responsibilities ,  and  government  respon¬ 
sibilities. 

(6)  The  methodology  used  in  defining  the  failure  population  to 

be  used  during  demonstration  (considering  function,  component 
count,  component  failure  rate,  criticality  of  failure,  etc.). 

The  failure  population  chosen  should  maximize  the  information 
gained  on  the  testability  of  the  item. 

(7)  The  mechanism  for  inserting  failures  into  the  item  (e.g. , 
prefaulted  modules,  juzqper  wires,  simulated  failures  at 
module  pins,  etc.}. 

(8)  The  procedure  for  selecting  failures  from  the  demonstration 
failure  population  for  insertion  into  the  item. 

(9)  The  procedure  for  applying  stimulus  (e.g.,  operational  inputs, 
diagnostic  software  execution,  tester  stimulus,  etc.)  to  the 
faulted  item. 

(10)  The  procedure  for  measuring  and  observing  the  test  results 
(error  branching,  tester  measurements,  etc.). 

(11)  A  listing  of  all  support  items  required  to  insert  failures, 
apply  tests,  and  observe  responses. 
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(12)  A  methodology  for  the  Interpretation  and  analysis  of  results 
(i.e. ,  failure  detected  with  isolation,  failure  detected 
without  isolation,  failure  not  detected,  etc.). 

(13)  The  establishment  of  acceptance  criteria  for  the  demonstration 
(%  fault  coverage^  degree  of  fault  isolation,  time  to  test, 
false  alarms)  in  accordance  with  MIL-STD-XXX 

(14)  A  plan  for  taking  corrective  action  as  may  be  needed. 

(15)  The  documentation  of  demonstration  results  in  the  Testability 
Analysis  Report. 


3.  Completion  Criteria. 

This  task  is  completed  when  the  government  has  agreed  to  the  details  as 
set  forth  in  the  plan  and  has  accepted  the  Testability  Demonstration 
Flan  in  accordance  with  the  terms  of  the  CDRL. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F5B 


PHASE:  FULL  SCALE  DEVELOPMENT 

FUNCTION:  Testability  Demonstration 


TASK  TITLE: 


Conduct  Testability  Demonstration 


TASK  OBJECTIVE: 


Demonstrate  achievement  of  the  testability  design  goals. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


e  Design 
Engineering 

•  Application 
Software 

e  Support 
Equipment 


e  HW  BIT  Data,  T  Design  Data, 
HW  Availability  Schedule 


e  Demonstration  Test  HW 
Resource  Requirements 


e  sw  BIT  and  Diagnostic  Test  e  Demonstration  Test  SW 
Data,  SW  Availability  Schedule  Resource  Requirements 


e  Test  Equipment  Availability 
Schedule 


e  Test  Equipment 
Requirements 


e  Test 

Engineering 


e  Test  Resource  Data 


e  Demonstration  Test 
Resource  Requirements 


e  Program 
Management 


e  Schedule 


e  Equipment  and  Personnel 
Requirements 


COST  TRADEOFF  INTER-RELATIONSHIP S :  Not  relevant  to  this  task. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

1. 1  Testability  Demonstration. 

The  task  requirement  is  to  conduct  a  Testability  Demonstration  as  required 
by  the  provisions  of  the  contract  and  to  demonstrate  the  achievement  of 
specified  (or  goal)  testability  parameters  on  major  end  items. 

2.  Implementation . 

Implementation  should  be  carried  out  in  accordance  with  the  testability 
demonstration  plan  prepared  as  described  in  Full  Scale  Development  Task 
15.  The  demonstration  plan  should  contain  the  method  for  conducting  the 
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demonstration  test,  the  list  of  skills  required  to  run  the  tests,  th  > 
tasks  pool,  the  corrective  action  plans,  and  the  support/ test  equipment 
necessary  for  the  test. 

Results  should  include  records  of  the  test  data  on  the  testability  demon¬ 
stration  data  sheets  (see  Table  1).'^  Calculate  the  testability 
parameters  from  the  data  recorded  on  the  data  sheets.  Compare  the 
calculated  parameters  against  the  specified  parameters  to  test  for 
acceptability  test  data. 

Combining  the  Testability  demonstration  with  the  Maintainability  demonstra- 
,  tion  may  prove  cost-  and  schedule-effective.  It  is  also  prudent  to  dry- 
run  the  procedures  prior  to  formal  demonstration. 


2.1  Demonstration  Process. 

The  Testability  Demonstration  should  be  conducted  in  an  environment  which 
simulates,  as  closely  as  practicable,  the  test  environment  planned  for 
the  item.  This  environment  should  be  representative  of  the  support  equip¬ 
ment,  facilities,  and  technical  data  that  would  be  required  at  the 
maintenance  levels  defined. 

2.2  Built-in  Test. 

Each  task  is  demonstrated  by  inserting  the  failure  condition,  running 
operational  sequences  and/or  self-test  sequences,  and  recording  the 
results : 

a.  Each  task  should  be  analyzed  to  determine  whether  or  not  a  clear 
indication  of  equipment  failure  is  provided. 

b.  Each  task  should  be  analyzed  to  determine  the  level  of  ambiguity 
to  which  the  equipment  built-in  test,  external  test  equipment  or 
manual  test  procedures  perform  the  initial  isolation.  The  results 
should  be  entered  on  the  Testability  Demonstration  Data  Sheet. 

2.3  External  Test. M 

Modules  identified  by  BIT  as  having  possible  failures  should  be  removed 
from  the  prxme  equipment  and  exercised  on  external  tester (s): 

a.  Fach  module  test  should  be  analyzed  to  determine  whether  or  not 
a  correct  PASS/FAIL  indication  is  prtvided. 

b.  Each  correctly  FAILING  test  should  be  analyzed  to  determine  the 
level  of  ambiguity  to  which  the  test  performs  secondary  isolation. 

The  results  should  be  entered  on  the  Testability  Demonstration 
Data  Sheet. 
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2.4  Testability  Demonstration  Data  Sheet. ^ 

Entries  are  made  on  the  Testability  Demonstration  Data  Sheet  (Table  1) 
as  follows: 

Column  1  is  assigned  a  sequential  demonstration  task  number. 

Column  2  indicates  the  demonstration  task  pool  number  selected  at 
random  for  this  task.  This  number  is  the  link  to  the  detailed 
information  on  the  simulated  failure. 

Column  3  indicates  that  the  failure  did  (Y)  or  did  not  (N)  cause 
observable  and  documented  anomalies  in  the  operational  behavior 
of  the  system. 

Column  4  indicates  that  the  failure  was  (Y)  or  was  not  (N)  detected 
by  built-in  test  features. 

Column  5  indicates  that  the  module  containing  the  failure  failed  as 
a  UUT  on  the  ATE  and  all  other  failure-free  modules  passed  (Y) ; 
otherwise,  (N) .  The  ATE  testing  may  be  assisted  by  built-in 
test  features  within  the  module. 

Column  6  indicates  the  size  of  the  ambiguity  group  (in  equipments/ 
CIs)  resulting  from  system  test  localization.  If  the  system  did 
not  exhibit  anomalous  behavior  (column  3  =  .1) ,  column  6  is 
assigned  a  dash  for  "not  applicable.”  1*  the  system  exhibited 
anomalous  behavior  but  did  not  provide  any  localization  informa¬ 
tion,  column  6  is  assigned  an  N. 

Column  7  indicates  the  size  of  the  initia'  ambiguity  group  (in 
modules)  resulting  from  successful  isolation  by  equipment-level/ 
module-level  BIT.  If  BIT  did  not  successfully  isolate  the  failure, 
column  7  is  assigned  an  N.  If  BIT  did  net  detec  the  failure 
(column  4  =  N) ,  column  7  is  assigned  dash  for  "not  applicable." 
(Alternatively,  column  7  indicates  the  number  of  jdules  which 
would  be  redplaced  in  reaching  the  failed  module  if  the  modules 
are  replaced  n-at-a-time  in  the  exact  order  given  in  the  organiza¬ 
tional-level  fault  dictionary.) 

Column  8  indicates  the  size  of  the  secondary  ambiguity  group  (in 
components)  resulting  from  successful  isolation  by  the  ATE  system. 
The  ATE  testing  may  be  assisted  Yy  module-level  BIT.  If  ATE  did 
not  successfully  isolate  the  failure,  column  8  is  assigned  an  N. 

If  ATE  did  not  detect  the  failure  (column  5  =*  N) ,  column  8  is 
assigned  a  dash  for  "not  applicable."  (Alternatively,  column  8 
indicates  the  number  of  components  which  would  be  replaced  in 
reaching  the  failed  component  if  the  component?  are  replaced  n-at- 
a-time  in  the  exact  order  given  in  the  fault  dictionary. ) 

Column  9  is  the  time  required  for  failure  isolation  by  BIT,  measured 
from  the  initiation  of  BIT  to  the  correct  determination  of  the 
module  ambiguity  group. 

Column  10  is  the  time  required  for  failure  isolation  by  ATE, 
measured  for  each  module  from  the  initiation  of  testing  to  the 
correct  determination  of  the  component  ambiguity  group. 

Column  11  is  reserved  for  comments,  including  references  to 
deficiencies  and  corrective  actions. 
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2.5  Testability  Parameter  Calculations.^ 

The  following  testability  parameters  can  be  calculated  from  the  data  in 
the  Testability  Demonstration  Data  Sheet: 

PDS  =  proportion  of  sairple  failures  which  are  detected  by  the 
operational  system 

N 

*  PS  =  number  of  Y's  in  column  3 
T  total  failures  inserted 


PDB  =  proportion  of  sample  failures  which  are  detected  by 
built-in  test 

N 

*  PB  “  number  of  Y's  in  column  4 
T  T 


PQ  =  proportion  of  sanqple  failures  detected 


N 

D  =  number  of  Y's  in  column  3  or  4 
T  T 


P 


IS 


proportion  of  sample  system  detections  which  reduce 
equipment/CI  ambiguity  size 

number  of  "numbers"  (non-dash, non-N)  in  column  6 


PIB  “  proportion  of  sample  BIT  detections  which  reduce 
module  ambiguity  size 

-  number  of  "numbers"  in  column  7 


Pn.  ■  proportion  of  sample  failures  which  are  detected 
by  ATE 

■  NDA  =  number  of  X's  in  column  5 
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=  proportion  of  sample  ATE/module-level  BIT  detections 
which  reduce  component  ambiguity  size 

*  number  of  "numbers"  in  column  8 


Ig (K)  “  sampled  degree  of  localization  to  K  equipments  provided 
by  the  operational  system 


count  of  times  that  column  6 
nds 


tyi.)- 


sampled  degree  of  localization  to  L  or  fewer  equipments 
provided  by  the  operational  system 


I 

K«1 


VK> 


I  (K)  ■*  sampled  degree  Oi.  isolation  to  K  modules  provided  by 
built-in  test 


count  of  times  that  column  7  «=  K 


N. 


DB 


I '  (L)»  sailed  degree  of  isolation  to  L  or  fewer  modules  provided 
by  built-in  test 


.X* 

K-l 


B 


(K) 


I  (K)  -  sampled  degree  of  isolation  to  K  components  provided  by 
A  ATE  (and  module  BIT) 


count  of  times  that  column  8  •  K 
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IA' (L) *  sampled  degree  of  isolation  to  L  or  fewer  components 
provided  by  ATE  (and  module  BIT) 

L 

I  VK> 

K-l 


I'  (10,  I '  (L) ,  and  I  •  (L)  are  calculated  for  each  value  of  L 

o  a  A 

for  which  there  is  a  fault  isolation  requirement  specified. 

2.6  Accept/Reiect  Criteria/ 

a.  Proportions .  The  observed  proportions  previously  calculated  should 

be  compared  against  the  corresponding  specified  parameter  or  against 
the  corresponding  parameters  predicted  by  the  testability  models, 
according  to  contract  requirements.  The  specified  or  predicted 
value  should  be  considered  to  be  achieved  if  the  observed  parameter, 
P,  and  the  specified,  P  ,  have  the  following  relationship 

P>P  -  2  «v/  — 

a  Cy  T 

where  T  is  the  total  number  of  relevant  tasks  in  the  sample  and  2 
is  the  confidence  level  coefficient  given  below. 


b.  Failure  isolation  times.  The  observed  failure  isolation  time  for 
built-in  test  for  each  demonstration  task  is  taken  from  column  9  of 
Table  I.  For  those  tasks  which  did  not  result  in  successful  isola¬ 
tion  of  the  failure,  the  failure  isolation  time  using  manual  trouble¬ 
shooting  methods  should  be  estimated  and  used  as  the  observed  time. 
Appropriate  methods  of  MIL-STD-471  are  used,  substituting  observed 
isolation  times  for  observed  maintenance  times,  to  determine  if  the 
specified  mew  (or  maximum)  failure  isolation  time  is  mett 

Mew  specified:  Use  method  1 

Percentile  specified:  Use  method  2 

Both  specified:  Use  method  8 


i 
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2.7  False  Alarm  Rate  Demonstration  Criteria. ^ 


The  False  Alarm  Rate  Demonstration  should  be  based  qpon  the  criteria  of 
MIL-STD- 781,  Reliability  Tests.  The  MIL-STD-781  test  plan  to  be  used 
should  be  based  upon  the  time  available  for  test,  acceptable  decision 
risks,  and  specified  discrimination  ratio.  The  cumulative  period  of 
operating  time  (T)  must  as  a  minimum,  include  the  operating  time  duration 
of  the  reliability  demonstration  test(s).  Each  confirmed  false  alarm 
should  be  treated  as  a  relevant  failure.  The  specified  false  alarm  rate 
should  be  input  to  the  test  plan  as  the  Predicted  MTBF.  The  equipment 
should  be  considered  acceptable  with  respect  to  false  alarm  rate  if  the 
acceptance  criteria  of  the  selected  MIL-STD-781  test  plan  is  met. 

a.  Related  false  alarms.  If  two  or  more  observed  false  alarms  are 
positively  attributable  to  a  single  design  problem  which  has  been 
identified  for  correction,  only  one  false  alarm  is  chargeable  to  the 
false  alarm  count. 


2.8  Documentation . 

The  results  of  the  demonstration  should  be  documented  for  submittal  to 
the  government. 


2.9  Testability  Demonstration  Pass/Fall. 

Successful  completion  of  the  demonstration  is  indicated  if  a  comparison 
made  between  the  parameters  calculated  from  the  test  data  and  the  con¬ 
tractually  required  parameters  shows  that  the  accept  criteria  have  been 
met. 


3.  Completion  Criteria. 

The  task  is  complete  upon  government  acceptance  and  approval  of  the 
demonstration  report. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F6A 

PHASE:  FULL  SCALE  DEVELOPMENT 

FUNCTION:  Final  Testability  Analysis  Report 


TASK  TITLE:  Prepare  Final  Testability  Analysis  Report 

TASK  OBJECTIVE:  Provide  a  single-document  repository  for  all  significant 

testability  data. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  Design 

i  HH  T  Design  Data,  BIT  data. 

e  Testability  Analysis 

Engineering 

HW  Tradeoffs 

Report 

e  Application 

e  SW  T  Features,  SW  Tradeoffs 

Software 

e  Support 

e  ATE  Interface  Data 

Equipment 

e  Test 

e  Test  Program  T  Data 

Engineering 

e  Life  Cycle 

e  Tradeoff  Costing 

Costing 

Data 

COST  TRADEOFF  INTER-RELATIONSHIPS :  Not  applicable  to  this  task. 


TASK  SYNOPSIS: 

1.  Task  Requirements 
1.1  Report  Data. ^ 

Prepare  a  Final  Testability  Analysis  Report  that  documents  the  testability 
program  instituted  and  used.  Specification  data,  tradeoffs/analyses 
results,  demonstration  data,  and  standing  recommendations  should  be 
included  within  the  body  of  the  report,  with  cross-reference  to  source 
and  substantiating  data  and  documents.  The  testability  analysis  report 
should  include  the  following  information.  (Previously  submitted  material 
which  is  unchanged  may  be  referenced  or  its  location  identified.) 
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Testability  Specifications.  System/Cl  testability  specifications 
approved  at  PDR  and  CDR,  plus  any  changes  approved  during  Full  Scale 
Development. 

Testability  Tradeoffs.  Documentation  of  the  impact  of  operational 
requirements  on  testing,  alternative  designs,  tradeoff  analysis,  and 
rationale  for  selection  of  design  alternatives.  Includes  data  pre¬ 
sented  at  PDR,  and  CDR,  plus  any  new  analysis  due  to  changes  during 
Full  Scale  Development. 

Qualitative  Testability  Analysis.  The  material  submitted  in  the 
preliminary  Testability  Analysis  Reports  at  PDR  and  CDR,  updated  to 
reflect  any  functional  design  changes  during  detail  design. 

Quantitative  Testability  Analysis,  ATE  Testing.  The  UUT  testability 
analysis  data,  for  each  type  of  Unit  Under  Test,  including: 

(1)  Definition  of  the  failure  population  in  accordance  with  the 
specified  failure  modes. 

(2)  Identification  of  the  test  stimulus,  including  built-in 
stimulus  and  external  stimulus. 

(3)  Determination  of  the  percentage  of  failures  in  the  failure 
population  which  are  detected  by  the  test  stimulus. 

(4)  Determination  of  the  level  of  fault  isolation  achievable. 

(5)  Justification  for  classes  of  faults  remaining  undetected  or 

which  are  poorly  isolated. 

Quantitative  Testability  Analysis,  Built-In  Testing.  The  testability 
analysis  data  for  each  configuration  item  including: 

(1)  Definition  of  the  failure  population  in  accordance  with  the 
specified  failure  inodes. 

(2)  Identification  of  the  built-in-test  stimulus. 

(3)  Determination  of  the  percentage  of  failures  in  the  failure 
population  which  are  detected  by  the  test  stimulus. 

(4)  Determination  of  the  level  of  fault  isolation  achievable. 

(5)  Justification  for  classes  of  faults  remaining  undetected  or 

which  are  poorly  isolated. 

Quantitative  Testability  Analysis,  System  Level.  The  overall  test¬ 
ability  analysis  data,  including: 

(1)  A  description  of  the  integration  of  testing  for  each  item  at 
the  system  level. 

(2)  A  determination  of  the  overall  system  failure  coverage  and 
level  of  fault  isolation. 
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(3)  An  estimate  of  developmental  and  recurring  costs  associated 
with  design  for  testability. 

(4)  An  estimate  of  developmental,  production,  and  support  savings 
associated  with  design  for  testability. 

(5)  An  estimate  of  system  readiness  improvement  attributable  to 
design  for  testability. 

g.  Testability  Demonstration  Data.  Summary  data  for  each  item  involved 
in  testability  demonstration  including  original  plans,  summarized 
results  and  any  corrective  action  taken.  The  detailed  demonstration 
procedures  and  raw  test  results  need  not  be  included. 

h.  Recommendations .  Recommended  action  to  be  taken  to  remedy  test¬ 
ability  deficiencies  or  improve  the  level  of  testability  achievable 
through  prime  equipment  engineering  changes,  ATE  improvements  and/or 
Test  Program  Set  improvements . 


22  Implementation. 

The  preliminary  testability  analysis  completed  in  the  Validation  Phase 
(Task  Reference  Number  V5A)  contains  much  of  the  prelininary  data  needed 
for  the  final  report.  The  final  testability  analysis  report  is  arrived 
at  by  updating  the  initial  data,  then  incorporating  essential  information 
drawn  from  those  testability  tasks  completed  in  the  Full  Scale  Development 
Phase . 

2.1  Report  Preparation  Instructions. 

A  Data  Item  Description  (DID)  has  been  written  and  proposed  as  the  stan¬ 
dard  for  preparation  of  the  Testability  Analysis  Report.  The  following 
is  a  paraphrased  excerpt  from  that  DID. ^ 

a.  Recomnended  for  inclusion  in  the  qualitative  sections  of  the  Test¬ 
ability  Report  include: 

(1)  A  description  of  the  partitioning  used  to  enhance  testability 
in  accordance  with  MIL-STD-XXX. 

(2)  A  description  of  each  applicable  item  (i.e.,  potential  or 
actual  UUT) . 

(3)  An  analysis  of  potential  failure  modes  and  effects  for  each 
item.  Data  on  failure  rates  and  confidence  levels  may  be 
referenced  from  the  Reliability  Program,  as  is  applicable. 

(4)  A  summary  of  the  overall  maintenance  concept  taken  from  Main¬ 
tainability  Program  and  ILS  Plan.  A  description  of  the  overall 
test  strategy  to  imp lament  the  maintenance  concept,  including 
coordination  between  BIT  and  ATE. 
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(5)  A  description  of  the  test  strategy  to  be  used  for  each  applic¬ 
able  item,  as  determined  by  the  overall  test  strategy. 

(6)  A  functional  description  of  built-in-test  features,  including 
hardware  and  software  BIT,  and  testability  features,  including 
controllability  and  observability  considerations,  for  each  item. 

(7)  A  functional  description  of  testability  measurement  techniques 
to  be  used,  including  computer-aided  analysis  tools. 

b.  Recommendations  for  inclusion  in  the  quantitative  sections  of  the 

Testability  Analysis  Report  include: 

(1)  For  each  item  defined  in  2.1a(2)  above,  a  description  of  the 
Testability  Analysis  Model  including: 

•  A  definition  of  the  failure  population  in  accordance  with 
the  specification. 

•  Identification  of  the  test  stimulus,  including  built- in-test 
stimulus  and  external  stimulus. 

•  A  determination  of  the  percentage  of  failures  in  the  failure 
population  which  are  detected  by  the  test  stimulus. 

•  A  determination  of  the  level  of  fault  isolation  achievable 
with  the  test  stimulus. 

•  The  justification  for  classes  of  failures  remaining  unde¬ 
tected  or  which  are  poorly  isolated. 

(2)  For  the  overall  system: 

•  A  description  of  the  integration  of  the  items  and  their 
test  stimulus/response  at  the  system  level. 

•  A  determination  of  the  overall  system  fault  coverage  and 
level  of  fault  isolation  based  upon  an  appropriate  combina¬ 
tion  of  these  characteristics  for  each  item. 

•  An  estimate  of  developmental  and  recurring  costs  associated 
with  design  for  testability,  including  weight,  volume,  and 
reliabilJ cy  penalties,  and  increased  computer  memory  require¬ 
ments. 

•  An  estimate  >f  developmental,  production,  and  support  savings 
associated  with  design  for  testability,  including  reduced 
checkout  time,  training,  spares,  and  ATE  costs. 


c.  The  testability  analysis  report  shall  include  the  following  support 
data.  This  support  data  shall  be  included  within  the  body  of  the 
report. 
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The  Testability  Analysis  Support  Data  shall  reflect  each  item's 
current  configuration  and  include: 

(1)  Testability  Analysis  Model  for  the  item: 

e  Component  characteristics 

•  Component  failure  models 

•  Component  interconnection  data 

•  Test  control  nodes 

•  Test  observation  nodes 

e  Interrelationship  of  models 

(2)  Testing  Data  for  the  item: 

e  Test  stimulus 
e  Predicted  good  response 
e  Predicted  fault  responses 
e  Test  stimulus  restrictions 
e  Test  response  tolerances 
e  Fault  coverage  data 
e  Fault  isolation  data 


2.2  Completeness . 

The  Testability  Analysis  Report  may  be  contractually  required  by  CDRL/ 

DID.  The  degree  of  acceptability  can  be  enhanced  by: 

•  Adhering  to  the  DID  requirements 

•  Using  the  topic  checklist  under  2.1,  Report  Data,  as  a  gauge  of 
completeness 

e  Review  by  writing  services  quality  assurance  functions  of  the 

completed  report  for  adherence  to  writing  principles,  clarity  and 
conciseness . 


3.  Completion  Criteria. 

Successful  completion  of  this  task  occurs  upon  government  acceptance  and 
approval  of  the  Testability  Analysis  Report. 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F7A 


PHASE;  Full  Scale  Development 

FUNCTION;  Category /Developmental/Operational  Testability  Evaluation* 

TASK  TITLE;  Monitor/Evaluate/Propose  corrective  action 

TASK  OBJECTIVE;  Exploit  CAT/DT/OT  to  obtain  early  assessment  of 
testability  effectiveness. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIP; 


T  INTERFACE 

INPUT  TO  T 

OUTPUT  FROM  T 

e  Design 

• 

HW  Design  Test 

•  HW  Related  Corrective 

Engineering 

Requirements 

T  Action 

e  Application 

• 

SW  Design  Test 

•  SW  Related  Corrective 

Software 

Requirements 

T  Action 

e  Test  Engineering 

• 

Test  Data 

e  Reliability 

• 

Data  and  Consultation 

•  Data  and  Consultation 

Engineering 

•  Maintainability 

• 

Data  and  Consultation 

•  Data  and  Consultation 

Engineering 

•  Life  Cycle 

• 

Proposed  Corrective 

•  Testability  Data 

Costing 

Action  Cost  Tradeoffs 

Supporting • LCC 

COST  TRADEOFF  INTER-RELATIONSHIPS;  The  Life  Cycle  Costs  associated  with 

corrective  actions  should  be  evaluated. 
The  analysis  required  is  largely  one  of  costs  and  penalties  of  implementing 
change,  compared  to  cost  savings  and  benefits  realized  from  the  change. 

*  The  major  source  document  for  this  task  used  the  term  "Operational  Test 
and  Evaluation"  which  is  the  term  also  used  in  the  task  synopsis.  The 
term  may  be  construed  to  mean  also  Category  testing  or  Developmental 
Testing. 
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TASK  SYNOPSIS: 


1.  Task  Requirements . 

The  task  consists  essentially  of  data  access  and  evaluation. 

a.  Data  collection.  Establish  a  system  to  access  or  collect  the  data 
required  for  the  evaluation  of  testability  phenomena  occurring 
during  the  Operational  Test  and  Evaluation  of  the  system  or 
equipment.  Data  for  analysis  of  confirmed  failures  and  false 
alarms  is  of  high  significance. 

b.  Data  analysis.  Develop  a  methodology  for  analysis  of  the  opera¬ 
tional  effectiveness  of  the  testability  design. 

c.  Corrective  action.  Where  the  analysis  projects  future  likelihood 
of  non-attainment  of  testability  goals  or  other  problems  relating 
to  testability ,  develop  means  of  proposing  and  evaluating 
modifications  to  the  prime  equipment,  test  equipment,  software  or 
other  test  program  element  as  appropriate. 

2.  Implementation. ^ 

The  objective  of  the  testability  evaluation  is  to  evaluate  the  impact 
of  actual  operational  and  maintenance  environments  on  the  ability  of 
production  equipment  to  be  tested.  The  effectiveness  of  testability 
design  techniques  'or  intermediate  or  depot  level  maintenance  tasks 
should  be  monitored  and  evaluated  as  part  of  the  operational 
testability  evaluation.  The  maintenance  tasks  to  be  evaluated  should 
be  limited  to  those  resulting  from  actual  operational  problems  and 
maintenance  actions. 

2.1  Data  Collection. ^ 

A  procedure  to  collect  or  otherwise  access  data  must  be  established 
prior  to  the  operational  test,  preferably  as  an  adjunct  to  that  used 
for  data  collection  by  the  maintainability  group  or  by  the  reliability 
group.  The  data  collected  should  include  a  description  of  all  opera¬ 
tional  anomalies  and  maintenance  actions.  In  addition,  the  data  system 
should  be  compatible  with  existing  data  systems  in  use  by  the  user 
organization. 

a.  Tracking  of  confirmed  failures.  Data  for  each  confirmed  failure 
should  be  compiled  to  address  the  following  questions : 

(1)  Built-in  Test. 

(a)  Did  built-in  test  detect  the  failure? 

(b)  Did  built-in  test  correctly  indicate  which  mission 
functions  were  lost? 
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(c)  Did  built-in  test  provide  accurate  fault  isolation 
information  for  corrective  maintenance  actions? 


(d)  What  was  the  ambiguity  size  (number  of  modules  to  be 
removed  or  further  tested)  due  to  fault  localization/ 
isolation  by  built-in  test? 

(e)  How  much  time  was  required  for  fault  isolation  at  the 
organizational  level  of  maintenance? 

(2)  ATE  Testing. 

(a)  Were  any  workarounds  required  to  overcome  mechanical  or 
electrical  deficiencies  in  the  UUT/ATE  interface? 

(b)  Was  the  documentation  for  UUT  hookup  and  power-up 
procedures  accurate? 

(c)  Did  the  ATE  system  provide  failure  detection  results 
consistent  with  those  of  the  initial  failure  detection 
by  BIT? 

(d)  Did  the  ATE  system  (in  conjunction  with  any  module  BIT) 
provide  accurate  fault  isolation? 

(e)  Were  the  test  results  repeatable? 

(f)  Was  the  observed  test  result  documented  in  the  mainten¬ 
ance  documentation? 

(g)  Was  the  failed  component  listed  under  the  observed  test 
result  in  the  maintenance  documentation? 

(h)  What  was  the  ambiguity  size  (number  of  components  to  be 
removed  or  further  tested)  due  to  fault  isolation  by  the 
ATE  system? 

(il  How  much  time  was  required  for  fault  isolation? 

(j)  How  much  total  time  (calendar  time)  was  required  to  re¬ 
pair  the  module? 

b.  Tracking  of  false  alarms.  Data  collected  should  include  a 

description  of  all  cases  in  which  the  built-in  test  reported  the 
detection  of  a  failure  but  with  no  operational  anomalies  present 
and  with  no  faults  subsequently  identified.  The  data  should  also 
document  cases  in  which  an  ATE  test  program  declared  a  failure  in 
a  UUT  with  no  faults  subsequently  identified  within  the  UUT.  The 
data  should  be  grouped  by  alarm  type  and  address  the  following 
questions : 
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(1)  What  is  the  characterization  of  the  alarm  type? 

(2)  What  is  the  frequency  of  occurrence  of  the  alarm? 

(3)  What  failure  or  failures  are  expected  to  cause  the  observed 
alarm? 

(4)  What  are  the  potential  consequences  of  ignoring  the  alarm  (in 
terms  of  crew  safety,  launching  unreliable  weapons,  etc.)? 

(5)  What  are  the  operational  costs  of  responding  to  the  false 
alarm  (in  terms  of  aborted  missions,  degraded  mode  operation, 
system  downtime) ? 

(6)  What  are  the  support  costs  associated  with  a  false  alarm? 

(7)  What  additional  data  is  available  from  operational  software 
dumps?  (e.g.,  soft  failure  occurrences,  branch  histories, 
interrupt  histories,  etc.)? 

(8)  Has  the  system  environment  (or  the  understanding  of  the 
system  environment)  changed  since  the  system's  tolerances  or 
•transient  characteristics  were  specified? 


2.2  Analysis . 

The  literature  provides  little  specialized  guidance  for  analysis  of 
results,  and  the  implementation  is  that  of  generalized  problem  solving. 
The  basic  purpose  of  the  analysis  is  to  "develop  a  tailored  methodology 
for  the  analysis  of  operational  problems  so  as  to  ascertain  if  built-in 
test  hardware  and  software,  ATE  hardware  and  software,  and  maintenance 
documentation  are  meeting  specifications  in  terms  of  fault  coverage, 
fault  resolution,  false  alarms,  fault  detection  times  and  fault 
isolation  times.  The  testability  models*  and  analysis  should  be  used 
as  the  basis  for  evaluating  the  effectiveness  of  any  proposed  modifi¬ 
cations  to  prime  equipment  or  test  equipment."  The  following  hints 
for  conduct  of  analysis  are  drawn  largely  by  inference  from  related 
source  material. 

a.  Work  objectively  with  data  elements  accessed  or  collected  as 
noted  above. 

b.  Separate  testability  issues  from  those  which  are  more  properly  of 
a  reliability,  maintainability  or  other  nature;  conduct  liaison 
with  the  other  disciplines  to  assure  appropriate  treatment  of  all 
data. 


*See  Task  V4D  (Ed.) 
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c.  Ascertain  conditions  unique  to  the  test  environment  which  may 
influence  the  analysis  as  projected  to  the  true  future  operating 
environment . 

d.  Orient  the  analysis  to  determination  of  the  extent  to  which  the 
system  meets  the  T  requirements  and/or  goals  and  to  correction  of 
true  T  potential  problems  at  the  earliest  possible  time  to  provide 
the  least  complex  and  most  cost-effective  solutions. 

e.  Merge  data  and  analyses  with  the  parallel  results  from  contractor 
and  government  development  testing  and  from  testability,  reliabili¬ 
ty  and  maintainability  demonstrations. 

f.  Organize  and  document  the  presentation  of  analysis  results  to  be 
objective  and  to  lucidly  support  the  recommendations . 

g.  Use  the  guidance  contained  in  the  Testability  Task  Compendia  of 
functions  V4,  V5  and  V6  to  aid  in  structuring  the  analysis. 

h.  Refer  to  Appendix  A  for  excerpts  from  the  literature  which  may  aid 
in  false  alarm  analysis  and  BIT  effectiveness  evaluation. 

2.3  Corrective  Action. 


As  is  the  case  with  analysis,  corrective  action  implementation  relies  • 
on  general  problem  solving  techniques.  The  following  hints  are  drawn 
by  inference  from  related  source  material. 

a.  The  analysis  should  evolve  completely  to  the  identification  of 
optimum  solutions.  The  urge  to  correct  problems  by  re-design 
should  be  resisted  until  it  is  proven  that  no  other  alternatives 
exist. 

b.  Coordinate  each  recommendation  with  all  other  disciplines  affected 
by  the  specifics  of  the  recommendation. 

c.  Include  with  each  recommendation  a  cost  analysis  treating  costs 
and  benefits  for  the  entire  life  cycle. 

3.  Completion  Criteria. 

This  task  may  have  indefinite  closure  where  its  reconmendations ,  in 
whole  or  in  part,  require  long  term  action  plans.  The  task  may  be 
considered  as  completed  after  government  approval  of  the  analysis 
report  and  recommendations  and  after  assumption  of  responsibility,  by 
a  management  entity,  for  any  remaining  testability-related  action  items. 
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APPENDIX  F7A1 


False  Alarm  Causes. 


(15) 


False  alarms  occur  for  several  reasons :  a)  the  system  under  test  does 
not  operate  as  expected,  b)  the  test  mechanization  is  incorrect, 
c)  the  software  is  programmed  incorrectly,  d)  failure  limits  have 
been  set  too  tight  for  dynamic  conditions ,  e)  test  system  logic 
circuitry  fails,  or  f)  test  parameters  are  affected  by  noise. 

. these  false  indications  .  (require)  detailed  investigation  of 

the  system  performance  and  coordination  with  the  design  personnel, 

system  supplier,  and  . test  personnel  to  effect  an  optimum 

resolution. 


2.  A  BIT  Evaluation  Technique. ^ 16 ^ 


BIT  evaluation  may  be  aided  by  use  of  the  following  technique,  which 
is  based  on  a  variation  of  a  simple  truth  table. 

TABLE  1 


BIT  INDICATION  MATRIX 


BIT 

INDICATION 

ACTUAL 

FAILURE 

NO 

FAILURE 

- > 

(correct) 

(false) 

NO  -  GO 

a* 

c* 

- ». 

(false) 

(correct) 

GO 

b* 

a* 

*The  letters  are  notations  for  terms  used  in  the  following 
equations : 

(1)  Fault  Detection  (FD) :  The  MIL-STD-1309B  definition  of  fault 

detection  is  -  "One  or  more  tests  performed  to  determine  if  any 
malfunctions  or  faults  are  present  in  a  unit."  In  equation  form 
and  relating  back  to  Table  1,  detection  as  a  measure  of  BIT  effec¬ 
tiveness  can  be  expressed  as: 


BIT/FT  (%) 


a  +  b 


X  100% 
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(2)  False  Alarm  Rate  (FAR) :  The  MIL-STD-1309B  definition  of  false 
alarm  is  -  "An  indicated  fault  where  no  fault  exists."  In 
equation  form,  FAR  can  be  expressed  as: 

BIT/FAR  (%)  -  ~  X  100% 


(3)  Fault  Isolation  (FI) ;  The  MIL-STD-1309B  definition  of  fault 

isolation  is  -  "Tests  performed  to  isolate  faults  within  the  unit 
under  test,"  and  is  an  equation  for  a  given  sample  it  is: 

BIT/FI  (%)  =  -Sprrectiy  isolatedjgf.  BIT  x  1Q()% 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  F8A 


PHASE: 


FUNCTION: 


TASK  TITLE: 


FULL  SCALE  DEVELOPMENT 


Final  Preproduction  Readiness  Review 


Provide  field  testability  data 


TASK  OBJECTIVE:  Assure  that  optimum  testability  is  contained  in  production 

designs  and  that  timely  actions  are  continuing  to  resolve 
outstanding  corrective  actions. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 


Coordination  is  appropriate  with  all  other  disciplines  participating  in  the 
program.  The  inter-relationships  are  those  of  all  the  functions  and  tasks 
of  the  FSD  phase. 

COST  TRADEOFF  INTER-RELATIONSHIPS: 

Underlying  inter-relationships  are  those  of  all  the  functions  and  tasks  of 
tiv  ^SD  phase  which  are  relevant  to  any  action  items  still  in  resolution  or 
completion  stages  at  the  time  of  the  final  review. 


TASK  SYNOPSIS: 

1.  Task  Requirements . 

1.1  This  review  may  be  formal  or  informal,  serving  if  needed  to  support  CDR- 
type  activity  (where  the  system  readied  for  production  is  in  turn  a 
subsystem  of  a  larger  or  more  comprehensive  system)  or  simply  for  internal 
recapitulation  of  the  testability  design  for  benefit  of  the  testability 
participants  only.  It  might  also  support  DSARC.  (or. lower  level  SARC) 
activity. 


2.  implementation . 

The  concept  for  implementation  is  the  same  as  that  for  Task  14  (CDR 
Support),  i.e.,  preparation,  presentation  and  followup. 

2.1  Preparation. 

Preparation  for  the  review  consists  of  review  of  all  testability 
materials  and  data,  review  of  the  testability  data  collected  during  the 
operational  test  and  evaluation,  preparation  of  recommendations  to 
improve  testing,  and  organization  of  the  major  testability  elements  for 
concise,  unambiguous  presentation . 
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2 . 2  Presentation . 


The  testability  engineer  may  present  summary  information  from  the  test¬ 
ability  analysis  report,  from  the  testability  demonstration,  and  from  the 
operational  test  and  evaluation.  All  necessary  support  data  must  also  be 
prepared  for  possible  review. 


2.3  Participation . 

The  testability  engineer  must  also  participate  in  the  review,  revision, 
and  approval  process  of  the  overall  readiness  review. 


3*  Completion  Criteria. 


3.1  Government  approval  of  the  Testability  Analysis  Report  and  the  system/ 
equipment  specifications,  together  with  approval  of  review  minutes 
comprise  the  measure  of  completion.  Action  item  responsibility  may  be 
transitioned  to  another  form  of  continuing  management  attention. 
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FULL  SCALE  DEVELOPMENT  PHASE  BIBLIOGRAPHICAL  REFERENCES 


1.  A  Study  of  Testability  Standardization  for  Electronic  Systems  and 
Equipment  prepared  by  Testing  Technology  Office,  NSVIC  &  Command  & 

Control  Applications  Branch,  NOSC  for  NESC,  Code  304,  pg.  20, 

Appendix  B,  37. 

2.  The  Test  Requirements  Document  A  Key  to  Improved  Electronic  Support  by 
E.E. Johnson,  Jr,.;  Indus try /Joint  Services  Automatic  Test  Conference  & 
Workshop  Proceedings,  San  Diego,  Apr  78,  pg.  363. 

3.  MIL-STD-1519 (USAF) ,  Test  Requirements  Document,  Preparation  of. 

NOTICE  1,  1  August  1977. 

4.  Dt-E-30139  Computer  Program  Development  Specification. 

5.  Test  Programming  Languages  by  Damon  C.  Hart;  Electronics  Test,  June 
1980,  pg.  48. 

6.  MIL-STD-2077 (AS)  Test  Program  Sets,  General  Requirements  for. 

9  March  1978 

7.  Cost  Estimating  Relatio;  ships  for  Automatic  Test  Equipment  Test  Program 
Set  Development  by  P.M.  Toscano;  Industry/Joint  Services  Automatic 
Test  Conference  &  Workshop  Proceedings,  San  Diego,  Apr  78,  pg.  366.  „ 

8.  Selection  Guide  for  Digital  Test  Program  Generation  Systems, 

NAVMATINST  3960. 9A  dtd  19  September  1979. 

9.  Life  Cycle  Costs  of  Test  Program  Sets  by  D.J.  Zingg;  Indus try/ Joint 
Services  Automatic  Test  Conference  &  Workshop  Proceedings,  San  Diego 
Apr  78,  pg.  458. 

10.  Small  Digital  Module  Testers  for  Shipboard  Use  (DIMOTE)  by  G.  Margulies, 
NAVELEX  4804;  Final  Report  Usability  and  Cost  Effective  Selection  of 
Test  Equipment  prepared  for  Naval  Electronics  Systems  Engineering  Center 
San  Diego,  CA,  Contract  No.  N0Q123-75-C-0910. 

11.  Economics  of  Failure  Detection  in  ATE  by  K.R.  Hilberth,  GT&T  Industries, 
Inc.,  Van  Nuys,  CA;  Industry /Joint  Services  Automatic  Test  .Conference 

&  Workshop  Proceedings,  San  Diego,  Apr  78,  pg.  513. 

12.  Digital  Automatic  Test  Program  Generation  Selection  by  D.B.  Day,  Support 
Systems  Associates,  Inc.;  CH  1488-6/79/0000-0295  @  1979  IEEE,  pg.  295. 

13.  Availability/Operational  Readiness  Test  Subsystem  Cost  Tradeoffs , 
RADC-TR-80-182,  pg,  37. 
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14.  A  Framework  for  Designing  Testability  into  Electronic  Systems  by  W.L. 
Keinerj  TR3826,  Naval  Surface  Weapons  Center,  Dahlgren  Laboratory 
(Code  K43),  Dahlgren,  VA  22448,  pg.  42,  53. 

15.  Onboard  Test  System  Design  Guide,  Rockwell  International,  North  American 
Division,  Los  Angeles,  CA;  TFD- 80-206. 

16.  BIT/SIT  Improvement  Project  (Phase  1) :  Evaluation  of  Selected  USAF 
Aircraft  BIT/SIT  Systems/Subsystems,  ASD-TR-79-5013 ,  pg.  47,  49. 
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TESTABILITY  TASK  COMPENDIA 
SECTION  IV  -  PRODUCTION  PHASE 

TABLE  OF  CONTENTS 

Task  Reference 

N»mher _  Task  Title 

P1A  Monitor  the  Production  Process  and 

Trends,  Review  Change  Proposals 

Bibliography  (Production  Phase) 

LIST  OF  ILLUSTRATIONS 

Production  Phase 


Page 

IV-3 

IV-9 

IV- 2 


IV-1 


LEGEND: 


FUNCTION 


V3 


C  2 


TASK  TITLE(S) 


FUNCTION 

NUMBER 


RELATED  FUNCTIONS 


Production  Phase  Testability  Tasks 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  PlA 

PHASE:  PRODUCTION 

FUNCTION:  Monitor/Evaluate  Testability  Data 

TASK  TITLE:  Initiate  and/or  review  change  proposals 

TASK  OBJECTIVE:  Assure  that  testability  is  taken  into  account  in  the 

change  proposal  cycle. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE _ INPUT  TO  T _ OUTPUT  FROM  T _ 

•  Maintainability  e  Maintainability  Evaluation  e  Evaluation  of  Data, 

Engineering  Data  Suggestions  for  T 

Improvements,  Initiation 

•  Producibility  •  Manufacturing  Data  of  Engineering  Change 

Engineering  Proposals  as  Required 

e  Test  Engineer-  e  Test  Results  Data 

ing,  Factory  & 

Customer 

•  Training  e  T  Relevant  Data 

e  Change  Review  e  Non-T  ECP  for  Evaluation  e  T  ECP  for  Evaluation 

Board 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Production  analysis  may  involve  any  of 

the  cost  tradeoffs  encountered  in  the  development  phases.  Actual  or  accrued 
cost  data  should  be  accessed  whenever  available.  Parameters  used  for  future 
cost  estimating  should  be  kept  current  in  the  light  of  observations  made  of 
actual  production. 


The  task  is  two-fold:  first,  to  sustain  the  integrity  and  consistency  of 
the  system 's  testability  characteristics  in  the  process  of  implementing 
ECP 1 s  and,  secondly,  to  enhance  testability  where  feasible  and  cost  effec¬ 
tive  via  the  ECP  process.  Access  to  or  collection  of  factual  data  is 
necessary  to  perform  analysis  which  objectively  support  all  comments  and 
recommendations . 
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Implementation . 

Implementation  consists  of  data  collection,  reduction  and  analysis 
followed  by  identification  of  corrective  actions.  The  testability 
engineer's  recommendations  for  courses  of  action  need  to  be  supported  by 
objective  conclusions  drawn  from  the  data  analysis  as  do  his  comments 
on  those  courses  endorsed  by  other  disciplines. 


2.1  Data  Collection.^ 

a.  From  manufacturing  processes,  cognizance  can  be  maintained  of  failures 
of  assembled  equipment  during  factory  and/or  acceptance  testing 
throughout  the  production  contract.  At  each  failure  occurrence, 
exercise  of  the  built-in-test  function  as  applicable  can  provide, 
compilations  of  the  following  data: 

(1)  degree  of  compliance  with  the  requirements  for  failure 
detection  and  isolation. 

(2)  description  of  the  malfunction  including  failure  synptoms. 

(3)  built- in-test  role: 

e  did  BIT  detect  the  failure? 

e  did  BIT  correctly  indicate  the  functions  lost? 
e  did  BIT  correctly  isolate  the  fault? 

•  what  was  the  ambiguity  size  due  to  BIT  fault  isolation? 

(4)  description  of  all  cases  in  which  BIT  reported  a  failure  but 
no  faults  were  subsequently  identified  (false  alarms) . 


b.  From  developmental  tests  and  operational  tests,  a  testability  data 
collection  system  can  be  established  that  is  integrated  as  much  as 
possible  with  similar  requirements  (such  as  reliability  and  main¬ 
tainability)  and  is  compatible  with  existing  customer  data  systems. 
Data  can  be  compiled  for  each  confirmed  failure  that  addresses  the 
following  questions: 

(1)  Built-in-test  role: 

e  did  BIT  detect  the  failure? 

e  did  BIT  correctly  indicate  which  mission  functions  were 
lost? 

e  did  BIT  provide  accurate  fault  isolation  information  for 
corrective  maintenance  actions? 

e  what  was  the  ambiguity  size  (number  of  modules  to  be 
removed  or  further  tested)  due  to  fault  localization/ 
isolation  by  BIT? 

e  how  much  time  was  required  for  fault  isolation  at  the 
organizational  level  of  maintenance? 


(2)  ete  (including  ATE)  Tasting 

a>  wars  any  workarounds  required  to  overcome  mechanical  or 
electrical  deficiencies  in  the  UUT/ETE  interface? 

e  was  the  documentation  for  UUT  hookup  and  power-up  pro¬ 
cedures  accurate? 

e  did  the  ETE  system  provide  failure  detection  results 
consistent  with  those  of  the  initial  failure  detection 
by  BIT? 

e  did  the  ETE  system  (in  conjunction  with  any  module  BIT) 
provide  accurate  fault  isolation? 

e  were  the  test  results  repeatable? 

e  was  the  observed  test  result  documented  in  the  maintenance 
documentation? 

e  was  the  failed  conponent  listed  under  the  observed  test 
result  in  the  maintenance  documentation? 

e  what  was  the  ambiguity  size  (number  of  components  to  be 
removed  or  further  tested)  due  to  fault  isolation  by  the 
ETE  system? 

e  how  much  time  was  required  for  fault  isolation? 

•  how  much  total  time  (calendar  time)  was  required  to  repair 
the  module? 

(3)  False  alarms 

•  what  is  the  characterization  of  the  alarm  type? 

e  what  is  the  frequency  of  occurrence  of  the  alarm? 

e  what  failure  or  failures  are  expected  to  cause  the 
observed  alarm? 

e  what  are  the  potential  consequences  of  ignoring  the  alarm 
(in  terms  of  crew  safety,  launching  unreliable  weapons,  etc)? 

e  what  are  the  operational  costs  of  responding  to  the  false 
alarm  (in  terms  of  aborted  missions,  degraded  mode  opera¬ 
tion,  system  downtime)? 

e  what  are  the  support  costs  associated  with  a  false  alarm? 

e  what  additional  data  is  available  from  operational  software 
dumps?  (e.g. ,  soft  failure  occurrences,  branch  histories, 
interrupt  histories,  etc.) 

e  has  the  system  environment  (or  the  understanding  of  the 

system  environment)  changed  since  the  system's  tolerances  or 
transient  characteristics  were  specified? 

From  demonstrations,  the  same  data  collection  system  developed  for 
tracking  factory  failures  (2.1. a)  may  be  used  during  formal  demon¬ 
strations  (Testability,  Reliability  and  Maintainability)  and 
Qualification  Testing. 

From  introduction  into  customer  inventory,  the  following  are  existing 
maintenance  data  sources: 


(1)  Formal  AFM  66-1  Maintenance  Data  Collection; This  system 
is  the  major  AF-wide  maintenance  data  collection  system.  It 
is  a  multi-purpose  computerized  data  collection  system  that  is 
designed  to  capture  manhours  expended  in  maintenance  actions, 
when  failures  were  discovered,  how  the  equipment  malfunctioned, 
type  of  repair  action  taken,  type  of  failure,  and  dates  of 
actions . 

(a)  Input  Data. 

•  AFTO  Form  350  Uteparable  Item  Processing  Tag) :  This 
tag  is  filled  out  by  the  maintenance  crew  and  attached 
to  any  LRU  that  is  removed  from  an  aircraft.  Informa¬ 
tion  included  on  this  tag  is  the  Work  Unit  Code  (WUC) , 
how  malfunctioned  code,  when  discovered  code  and  a 
brief  written  description  of  the  problem/malfunction. 
Information  contained  on  this  tag  provides  the  direct 
input  for  the  completion  of  the  AFTO  Form  349  in  the 
intermediate  shop. 

•  AFTO  Form  349  (Maintenance  Data  Collection  Record) ; 

This  form  is  filled  out  at  the  intermediate  shop  for 
every  LRU  pulled  from  an  aircraft  and  is  assigned  a  Job 
Control  Number  (JCN) .  Some  of  the  data  that  is  in¬ 
cluded  on  this  form  by  specific  codes  is  the  WUC  of  the 
equipment,  when  discovered,  how  malfunctioned,  action 
taken  and  type  maintenance  performed.  Also,  space  is 
provided  to  write  a  description  of  the  discrepancy  and 
corrective  action  taken.  The  specific  codes  entered 

on  the  form  are  the  direct  inputs  to  the  computer 
reports. 

(b)  There  are  a  number  of  possible  computer  outputs  of  the 

AFM  66-1  system.  Some  of  the  available  outputs  are: 

e  BLIS  (Base  Level  Inquiry  System) :  This  output  records 
maintenance  activities  of  a  particular  base  and  is  nor¬ 
mally  available  only  at  the  base.  The  BLIS  reports 
are  compiled  from  AFTO  Form  349* s  and  are  available  in 
various  formats. 

e  AFLC  D056  Reports:  These  are  computerized  reports 

processed  by  AFLC  that  are  intended  as  measures  of 
product  performance  and  maintenance  manhours  and  reflect 
areas  that  require  improvement.  AFLCR  66-15  outlines 
the  different  possible  reports  available. 


AFTO  Form  95  (Significant  Historical  Data) :  This 
optional  form  can  be  filled  out  as  a  history  of  main- 
_ tenance  actions  taken  for  a  particular  LRU  every  time 
the  LRU  comes  into  the  intermediate  shop  for  repair. 
Actual  information  displayed  can  vary  from  shop  to  shop. 
Normally,  records  are  maintained  by  JCN,  WUC  and  LRU 
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serial  number  and  include  actual  repair  actions/ 
remedies,  components  replaced,  adjustments  made.  Not 
Reparable  This  Station  (NRTS)  codes,  Could  Not 
Duplicate  (CND)  or  bench  check  retest  OK  (RETOK) 
occurrences  and  test  station  downtime.  When  maintained 
accurately,  these  records  provide  a  measure  of  the  FAR 
and  indicate  which  LRUs  were  removed  for  BIT  fault 
detection  or  for  other  reasons,  or  both. 


(2)  USN  Standard  Maintenance  and  Material  Management  System 
(3M) :  This  system  is  virtually  parallel  to  the  AFM  66-1 
system,  using  USN  form  numbers  and  notations. 

(3)  The  Army  Maintenance  Management  System  (TAMMS) :  The  TAMMS 
also  parallels  the  AFM  66-1  system. 


2.2  Data  Reduction  and  Analysis. 

a.  A  methodology  for  data  reduction  and  problem  analysis  relevant  to 
testability  should  be  developed  so  as  to  ascertain  if  built-in-test 
hardware  and  software,  ATE  hardware  and  software,  and  maintenance 
documentation  are  meeting  specifications  in  terms  of  fault  coverage, 
fault  resolution,  false  alarms,  fault  detection  time  and  fault  isola¬ 
tion  time. 

b.  The  testability  data  reduction  and  analysis  methodology  may  be 
extended  to  identify  trends  relevant  to  other  hardware  characteris¬ 
tics  such  as  a  change  in  component  failure  rate  or  increase  in  remove 
and  replace  times. 


2.3  Corrective  Action. 

The  data  collection,  reduction  and  analysis  activity  may  identify  test¬ 
ability  related  problems  whose  only  solution  is  by  EOF.  The  testability 
models  and  analyses  developed  during  the  validation  phase  and  updated 
during  full  scale  development  may  be  used  to  justify  the  proposed  modifi¬ 
cations  to  the  prime  equipment  or  test  equipment.  For  ECPs  generated  by 
groups  other  than  testability,  the  testability  models  and  analyses  may 
be  used  to  aid  in  evaluating  the  effectiveness  of  the  proposed  modifica¬ 
tion. 


3.  Completion  Criteria. 

The  overall  task  of  collecting  data,  analyzing  data  and  taking  corrective 
action  will  continue  into  the  operation  and  support  phase  activity  as 
demonstrations  and  testing  are  replaced  by  the  build-up  of  operating 
inventory. 
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The  task  may  be  considered  as  complete  when  production  has  been  completed 
and/or  the  government  assumes  engineering  responsibility  for  the  prime 
system.  Alternatively,  the  responsibility  may  transition  to  the  entity 
responsible  for  monitoring  of  operations  and  support  (Function  Dl) . 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  DlA 


PHASE: 


FUNCTION: 


TASK  TITLE: 


TASK  OBJECTIVE: 


OPERATION  AND  SUPPORT,. 

Monitor /Evaluate  Testability  Data 
Monitor  operation  &  support  activity 


Measure  the  achieved  testability  of  the  system  for 
improving  the  current  system  and  for  application  of 
Knowledge  to  new  and  evolving  designs. 

DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS: 

T  INTERFACE  INPUT  TO  T  OUTPUT  FROM  T 


e  Contractor 

e  Field  Data 

e  Testability  Trends, 

Field  Service 

Engineering  Change 

Proposals 

e  Government 

e  Field  Data 

e  Testability  Trends, 

Data  Systems 

Engineering  Change 

Proposals 

COST  TRADEOFF  INTER-RELATIONSHIPS:  Operations  and  Support  Analysis  may 

involve  any  of  the  cost  tradeoffs  encountered  in  development  phases.  Actual 
or  accrued  cost  data  should  be  accessed  wherever  available,  and  parameters 
used  for  future  cost  estimating  should  be  kept  current  in  the  light  of 
observations  made  of  actual  operations  and  support. 


TASK  SYNOPSIS: 

1.  Task  Requirements. 

To  measure  achieved  Testability,  it  is  necessary  to  collect  cr  access  in 
any  way  feasible  all  operational,  maintenance,  repair,  test,  and  support 
data  from  all  operations  and  maintenarce  levels  to  determine  the  effec¬ 
tiveness  of  testability  design  techniques  for  organisational,  intermediate 
and  depot  level  maintenance  tasks. 


1.1  Monitoring  also  provides  the  means  to  evaluate  the  impact  of  actual  opera¬ 
tional  and  maintenance  environments  on  the  testability  of  the  production 
equipment. 
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Implementation . 

2.1  A  data  collection  system  should  be  established  as  necessary  to  meet  the 
needs  of  the  testability  evaluation.  The  data  collected  should  include 
a  description  of  all  operational  anomalies  and  maintenance  actions.  Data 
collection  is  to  be  integrated  as  much  as  possible  with  similar  data 
collection  requirements  such  as  those  for  reliability  and  maintainability. 
In  addition,  the  data  system  should  be  compatible  with  existing  data 
systems  in  use  by  the  military  user. 


2.2  Tracking  of  Confirmed  Failures. ^ 

Data  for  each  confirmed  failure  should  be  compiled  to  address  the 
following  questions: 

a.  Built-in  Test. 

e  Did  built-in  test  detect  the  failure? 

e  Did  built-in  tesc  correctly  indicate  which  mission  functions 
were  lost? 

a  Did  built-in  test  provide  accurate  fault  isolation  information 
for  corrective  maintenance  actions? 

•  What  was  the  ambiguity  size  (number  of  modules  to  be  removed 
or  further  tested)  due  to  fault  localization/isolation  by 
built-in  test? 

e  How  much  time  was  required  for  fault  isolation  at  the  organiza¬ 
tional  level  of  maintenance? 


b.  ETE/ATE  Testing. 

e  Did  the  ETE/ATE  system  provide  failure  detection  results 

consistent  with  those  of  the  initial  failure  detection  by  BIT? 

e  Did  the  ETE/ATE  system  (in  conjunction  with  any  module  BIT) 
provide  accurate  fault  isolation? 

e  Were  the  test  results  repeatable? 

e  Was  the  observed  test  result  documented  in  the  maintenance 
documentation? 

e  Was  the  failed  component  listed  under  the  observed  test  result 
in  the  maintenance  documentation? 

e  What  was  the  ambiguity  size  (number  of  components  to  be  removed 
or  further  tested)  due  to  fault  isolation  by  the  ETE/ATE  system? 
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•  How  much  tine  was  required  for  fault  Isolation? 

•  How  much  total  time  (calendar  time)  was  required  to  repair 
the  module? 


c.  Tracking  of  false  alarms.  Data  collected  should  include  a 

description  of  all  cases  in  which  the  built-in  test  reported  the 
detection  of  a  failure  but  with  no  operational  anomalies  present  and 
with  no  faults  subsequently  identified.  The  data  should  also  docu¬ 
ment  cases  in  which  an  ATE  test  program  declared  a  failure  in  a  UUT 
with  no  faults  subsequently  identified  within  the  OUT.  The  data 
should  be  grouped  by  alarm  type  and  address  the  following  questions: 

e  What  is  the  characterization  of  the  alarm  type? 

e  What  is  the  frequency  of  occurrence  of  the  alarm? 

e  What  failure  or  failures  are  expected  to  cause  the  observed 
alarm? 

e  What  are  the  potential  consequences  of  ignoring  the  alarm  (in 
such  terms  as  crew  safety  and  launching  unreliable  weapons)? 

•  What  are  the  operational  costs  of  responding  to  the  false  alarm 
(in  terms  of  aborted  missions,  degraded  mode  operation,  or 
system  downtime)? 

e  What  are  the  support  costs  associated  with  a  false  alarm? 

e  What  additional  data  is  available  from  operational  software 
dumps?  (e.g.,  soft  failure  occurrences,  branch  histories, 
interrupt  histories,)? 

e  Has  the  system  environment  (or  the  understanding  of  the  system 
environment)  changed  since  the  system's  tolerances  or  transient 
characteristics  were  specified? 


2. 3  Existing  Maintenance  Data  Sources. 

a.  Formal  AFM  66-1  Maintenance  Data  Collection. This  system  is  the 
major  AF  wide  maintenance  data  collection  system.  It  is  a  multi¬ 
purpose  computerised  data  collection  system  that  is  designed  to 
capture  manhours  expended  on  maintenance  actions,  when  failures  were 
discovered,  how  equipment  malfunctioned,  type  of  repair  action  taken, 
type  of  failure,  and  dates  of  actions. 
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(1)  AFTO  Form  350  (Reparable  Itwn  Processing  Tag)  t  This  tag  la 
filled  out  by  the  maintenance  crew  and  attached  to  any  LRU  that 
ie  removed  from  an  aircraft.  Information  included  on  this  tag 
is  the  Work  Unit  Code  (WUC) ,  how  malfunctioned  code,  when  dis¬ 
covered  code  and  a  brief  written  description  of  the  problem/ 
malfunction.  Information  contained  on  this  tag  provides  the 
direct  input  for  the  completion  of  the  AFTO  Form  349  in  the 
intermediate  shop. 

(2)  AFTO  Form  349  (Maintenance  Data  Collection  Record:  This  form 
is  filled  out  at  the  intermediate  shop  for  every  LRU  pulled 
from  an  aircraft  and  is  assigned  a  Job  Control  Number  (JCN) . 

Some  of  the  data  that  is  included  on  this  form  by  specific 
codes  is  the  WUC  of  the  equipment,  when  discovered,  how  mal¬ 
functioned,  action  taken  and  type  maintenance  performed.  Also, 
space  is  provided  to  write  a  description  of  the  discrepancy  and 
corrective  action  taken.  The  specific  codes  entered  on  the 
form  are  the  direct  inputs  to  the  computer  reports. 

(3)  There  are  a  number  of  possible  conputer  outputs  of  the  AFM  66-1 
system.  Some  of  the  available  outputs  are: 

(a)  BLIS  (Base  Level  Inquiry  System) :  This  output  records 
maintenance  activities  of  a  particular  base  and  is  normally 
available  only  at  the  base.  The  BLIS  reports  are  compiled 
from  AFTO  Form  349* s  and  are  available  in  various  formats. 

(b)  AFLC  DO 56  Reports:  These  are  computerized  reports  pro¬ 

cessed  by  AFLC  that  are  intended  as  measures  of.  product 
performance  and  maintenance  manhours  and  reflect  areas 
that  require  improvement.  AFLCR  66-15  outlines  the 
different  possible  reports  available.  One  of  the  reports 
(the  D056B5006  report)  provides  equipment  historical  in¬ 
formation  on  the  maintenance  actions,  manhours,  and  aborts 
by  month  for  past  si.;  months  on  every  WUC  included  in  the 
master  record.  The  intent  of  the  report  is  to  show  trend¬ 
ing  and  performance  data  in  the  areas  of  failures, 
maintenance  actions,  manpower  resources  expenditure  and 
aborts. 

(4)  AFTO  Form  95  (Significant  Historical  Data):  This  optional  form 
can  be  filled  out  as  a  history  of  maintenance  actions  taken  for 
a  particular  LRU  every  time  the  LRU  cones  into  the  intermediate 
shop  for  repair.  Actual  information  displayed  can  vary  from 
shop  to  shop.  Normally,  records  are  maintained  by  JCN,  wuc  and 
LRU  serial  number  and  include  actual  repair  actiona/ramedies , 
components  replaced,  adjustments  made.  Not  Reparable  This 
Station  (NRTS)  codes.  Could  Not  Duplicate  (CND)  or  bench  check 
retest  OK  (RETOK)  occurrences  and  test  station  downtime. 

When  maintained  accurately,  these  records  provide  a  measure  of 
the  FAR  and  indicate  which  LRUs  were  removed  for  BIT  fault  detec¬ 
tion  or  for  other  reasons,  or  both. 
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(5)  USN  Standard  Maintenance  and  Material  Management  System  (3M) . 
This  system  is  virtually  parallel  to  the  AFM  66-1  system. 

(6)  The  Army  Maintenance  Management  System  (TAMMS) .  The  TAMMS 
also  parallels  the  AFM  66-1  system. 


2.4  Testability  Effectiveness  Measures 

Figure  1  offers  a  menu  of  techniques  for  measuring  testability 
effectiveness.  The  Figures  of  Merit  provide  a  point  of  departure  for 
analyses  which  might  pinpoint  precise  points  for  corrective  actions.  For 
any  particular  program,  a  set  of  FOM  might  be  selected  as  best  suited  to 
the  kinds  of  data  available. 


2.5  A  Sample  Technique  for  Measurement  of  BIT  Effectiveness. ^ 

Three  measures  of  BIT  effectiveness  (fault  detection  (FD) ,  fault  isola¬ 
tion  (FI)  and  false  alarm  rate  (FAR) )  have  been  offered  for  operational 
data  evaluation.  The  matrix  (Table  1)  shows  the  four  possible  BIT  con*' 
ditions  that  can  exist. 


TABLE  1.  BIT  INDICATION  MATRIX 


BIT 

INDICATION 

TRUE  EQUIPMENT  STATUS 

ACTUAL 

FAILURE 

1  NO 

FAILURE 

- > 

(correct) 

(false) 

NO-GO 

a 

c 

- > 

(false) 

(correct) 

GO 

b 

d 

By  interrelating  the  terms  of  Table  1,  the  three  measures  of  BIT  effec¬ 
tiveness  can  be  expressed. 

a.  Fault  Detection  (FD) : 

BIT  FD  (%)  -  a  X  100% 

a  +  b 
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b.  False  Alarm  Rate  (FAR) : 


BIT  FAR  (%)  -  c  X  100* 
a  +  c 

c.  Fault  Isolation  (FI) : 

BIT  FI  (*)  «  Faults  correctly  isolated  by  BIT  X  100% 

a 


Completion  Criteria. 

This  is  an  open-ended  task  that  may  be  conducted  at  one  or  more  intervals 
throughout  the  operational  life  of  the  system.  Testability  monitoring 
may  be  discontinued  at  such  time  as  it  is  established  that  the  testability 
goals  are  being  satisfactorily  achieved  in  view  of  the  remaining  antici¬ 
pated  life  of  the  system. 
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Figure  of  Merit  Definition  General  Model 
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TESTABILITY  TASK  COMPENDIUM 
Task  Reference  Number  DIB 


PHASE: 


Operation  and  Support 


FUNCTION : 


Monitor/Evaluate  Testability  Data 


TASK  TITLE: 


Review  change  proposals  for  testability 


TASK  OBJECTIVE: 


Maintain  or  improve  the  integrity  and  consistency  of 
testability  in  the  design  as  changes  and  retrofits  are 
developed. 


DESIGN  DISCIPLINE  TESTABILITY  (T)  INTER-RELATIONSHIPS : 


T  INTERFACE 


INPUT  TO  T 


OUTPUT  FROM  T 


•  Change  Control 
Board 


Proposed  Changes 


Testability  Evaluation 


Design 

Engineering 


Proposed  Changes 


Testability  Evaluation 


COST  TRADEOFF  INTER-RELATIONSHIPS:  Tradeoff  of  implementation  cost  penalties 

versus  future  cost  saving  benefits  is  an 
essential  step  in  evaluating  testability  (or  any  -ility)  candidates  for 
redesign.  Also  analysis  of  costs  associated  with  performance  improvements 
is  essential  to  the  normal  ECP  decision  processes.  Operations  and  support 
analysis  may  involve  any  of  the  cost  tradeoffs  encountered  in  development 
phases.  Actual  or  accrued  ^ost  data  should  be  accessed  wherever  available, 
and  parameters  used  for  future  cost  estimating  should  be  kept  current  in 
the  light  of  observations  made  of  actual  operations  and  support. 

TASK  SYNOPSIS: 

1.  Task  Requirements. 

The  data  collected  and  evaluated  in  Task  DlA  may  be  analyzed  ih  order 
to  propose  modifications  to  the  prime  equipment  or  test  equipment  as 
deemed  necessary  and  to  provide  testability  evaluation  of  all  proposed 
ECPs. 
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2. 


1 


Implementation . 

Two  types  of  ECPs  may  arise:  performance  or  other  considerations  may 
give  rise  to  change  proposals  originating  in  other  than  testability 
areas,  and  the  testability  monitoring  activity  may  wish  to  generate 
changes  to  improve  testability. 

2.1  The  testability  monitoring  activity  (contractor  and/or  customer) 
needs  to  establish  representation  on  the  change  review  (or  control) 
board  (contractor  and/or  customer)  in  order  to  have  a  voice  in  approval 
or  modification  of  proposals  to  change  the  system  where  the  changes 
impact  upon  testability  characteristics. 

2.2  The  monitoring  and  analysis  of  field  operations  and  maintenance  data 
may  identify  testability  problems  whose  only  solution  is  an  ECP,  e.g., 
a  change  to  a  test  program  tape  or  test  adapter.  Representation  on 
the  change  review  board (s)  is  not  essential  to  the  generation  of 
ECPs,  but  the  ECP  may  be  facilitated  where  such  representation  does 
exist. 

3.  Completion  Criteria. 

The  task  is  a  continuous  one  and  exists  for  as  long  as  there  is 
potential  for  ECP. 
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OPERATIONS  AND  SUPPORT  PHASE  BIBLIOGRAPHICAL  REFERENCES 


1.  A  Study  of  Testability  Standardization  for  Electronic  Systems  and 
Equipment  prepared  by  Testing  Technology  Office,  NSWC  and  Command  & 
Control  Applications  Branch,  NOSC  for  NESC,  Code  304,  pg.  37. 

2.  BIT/SIT  Improvement  Project  (Phase  1) :  Evaluation  of  Selected  USAF 
Aircraft  BIT/SIT  Systems/Subsystems,  ASD-TR-79-5013,  pg.  49. 

3.  BIT /External  Test  Figures  of  Merit  and  Demonstration  Techniques, 
RADC-TF-79-309 . 
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GLOSSARY  OP  TESTABILITY  TERMS 


The  sources  of  some  definitions  are  given  in  parentheses  following  the 
definition.  The  source  identifications  jure: 


ATG 


IEEE 


IEEE /PTC 

I/JS 

TR-3826 


Industry  ATG  Glossary,  Report  of  Industry 
Ad  Hoc  ATE  Project  for  the  Navy,  April  1977 

IEEE  Standard  Dictionary  of  Electrical  and  Electronics 
Terms,  IEEE  Std  100-1972 

Interim  IEEE  Technical  Committee  on  Fault  Tolerant 
Computing  Dictionary  of  Terms. 

Industry /Joint  Services  Automatic  Test  Project  Pinal 
Report 

A  Framework  for  Designing  Testability  Into  Electronic 
Systems 


MIL-STD-721B  Definition  of  Effectiveness  Terms  for  Reliability, 
Maintainability  Human  Factors  and  Safety 

MIL-STD-1309B  Definitions  of  Terms  for  Test,  Measurement  and 
Diagnostic  Equipment . 

MIL-STD-2077  Test  Program  Sets,  General  Requirements  for 


A 

Active  redundancy.  That  redundancy  wherein  all  redundant  items  are 
operating  simultanec  ly,  rather  than  being  switched  on  when  needed 
(IEEE) . 

Active  testing.  Closed- loop  testing  (TR3826) .  Testing  which  generates 
its  own  test  vectors. 

Ambiguity  group.  The  group  of  maintenance  replaceable  units  which  may 
contain  faults  which  result  in  the  same  fault  signature;  also  the  number 
of  units  in  such  a  group  (TR-3826) . 

ATE  bit  skew.  The  maximum  time  difference  between  the  first  and  last 
digital  pulse  arriving  at  the  ATE  interface  within  the  same  digital  test 
pattern . 

ATG  node.  An  ATG  system  may  translate  a  user's  unit  under  test  (UUT)  into 
a  logic-equivalent  circuit.  In  general,  the  translation  will  not  be  a 
one-to-one  mapping.  The  additional  nodes  thus  created  are  called  ATG 
node'-.  It  should  be  pointed  out  that  a  logic  equivalent  usually  does 
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not  have  one-to-one  correspondence  between  the  actual  hardware  of  the  UUT 
and  its  ATG  equivalent  circuit. 


Automatic  test  equipment  (ATE) .  Equipment • that  is  designed  to  conduct 
analysis  of  functional  or  static  parameters  to  evaluate  the  degree  of 
performance  degradation  and  that  may  be  designed  to  perform  fault 
isolation  of  unit  malfunctions.  The  decision  malting,  control,  or  evalua¬ 
tive  functions  are  conducted  with  a  minimum  reliance  upon  human 
intervention  (MIL-STD-1309B) . 

Availability.  A  measure  of  the  degree  to  which  an  item  is  in  the  operable 
and  commi table  state  at  the  start  of  the  mission,  when  the  mission  is 
called  for  at  an  unknown  (random)  point  in  time  (MIL-STD-721B) .  Avail- 
cibility  is  the  probability  of  a  system  readiness  over  a  long  interval  of 
time  (TR3826) . 

Average  isolation  time.  Average  time  to  follow  the  worst  case  logic 
chain  of  each  diagnostic  branch  containing  three  or  more  unique  tests 
(MIL-STD-2077  (AS) ) . 
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Back-tracing .  A  procedure  used  for  automatic  test  generation  which  ! 

starts  from  a  specified  node  and  traces  toward  its  source  or  sources 
until  primary  inputs  are  reached. 

Binary  simulator.  A  program  which  establishes  a  representation  of  a 
logic  circuit  or  configuration  based  upon  a  computer-di.rected  and/or 
processed  model  of  the  logic  circuit  or  configuration.  Node  points  and 
output  pins  of  the  simulated  circuit/configuration  are  permitted  to  take 
on  only  two  values --logical  1  or  logical  0 —  in  sequences  and  along  paths 
in  accord  with  program  rules,  in  order  to  derive  fault-detection  and 
fault-isolation  information  for  the  logic  circuit  or  configuration 
represented. 

bit  false  alarms.  A  false  alarm  occurs  if  a  component  is  declared 
defective  and  it  is  found  in  separate  off-line  test  to  be  failure  free. 

BIT  -  hardware  redundant.  Hardware  redundancy  refers  to  performance 
replication  where  functions  to  be  tested  are  mapped  from  the  operational 
circuitry  to  the  test  circuitry  essentially  on  a  1:1  basis.  Such 
standard  hardware  redundant  BIT  methods  may  be  further  partitioned  into 
continuous  and  sampled  fault  monitoring  methods.  Representative  time 
continuous  approaches  are  the  well-known  duplication,  triplication,  etc. 
approaches,  hon-continuous  BIT  hardware  redundant  approaches  may  be 
sampled  either  in  time  or  topologically  or  both. 

BIT  -  information  redundant.  Information  redundancy  refers  to  the  use  of  j 

coding  theory  techniques  such  as  those  widely  used  in  conmunications 
applications.  Coding  techniques  include  both  systematic  and  non-  i 

systematic  approaches.  Systematic  codes  are  those  where  the  encoded 
information  can  be  separated  from  the  data.  Non-systematic  codes  t 

combine  the  data  and  the  check  code  so  that  additional  processing  is  !  1 

required  to  separate  the  two. 

BIT  performance.  A  fad  lure  for  purposes  of  the  specification  means  that 
the  equipment  does  not  meet  the  requirements  of  the  procurement  specifi¬ 
cation  . 

J 

BIT  thoroughness.  The  BIT-detected  percent  of  all  failures  weighted  by 
failure  rates. 

Bridge  fault.  A  short  fault  (TR3826) .  Faults  caused  by  short  circuits 
between  adjacent  paths. 

Built-in  test  (BIT) .  A  test  approach  using  BITE  or  self-test  hardware  or 
software  to  test  all  or  part  of  the  unit  under  test  (OUT)  (MIL-STD-1309B) . 

Built-in  test  equipment  (BITE) .  Any  device  which  is  part  of  an  equipment 
or  system  and  is  used  for  the  express  purpose  of  testing  that  equipment  or 
system.  BITE  is  an  identifiable  unit  of  the  equipment  or  system 
(MIL-STD-1309B)  . 
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Casualty .  A  manifestation  of  a  failure  at  the  system  level  or  major  sub¬ 
system  level  such  that  the  system/subsystem  is  incapable  of  performing 
its  principal  function ( s) .  A  casualty  is  differentiated  from  a  malfunc¬ 
tion  by  the  greater  seriousness  or  persistence  of  its  nature  (TR3826) . 

Catastrophic  fault,  digital.  A  primary  failure  in  digital  circuitry  which 
causes  secondary  failures  (TR3826) . 

Catastrophic  fault,  analog.  A  fault  in  analog  circuitry  which  causes 
a  sudden  change  in  operating  characteristics  which  results  in  a  complete 
lack  of  useful  performance.  (ATG) 

Checkpoint.  A  place  in  a  routine  where  a  check,  or  recording  of  data  for 
restart  purposes,  is  performed  (IEEE). 

Closed- loop  testing.  Testing  in  which  the  input  stimulus  is  controlled  by 

the  equipment  output  monitor  (MIL-STD-1309B) . 

Comparison  tester.  The  utilization  of  a  known  good  board  ("golden  unit") 
as  a  means  for  comparing  test  results  with  the  unit  under  test  when  both 
are  subjected  to  the  same  stimuli. 

Component-internal  fault.  A  device  or  component  fault  which  is  not  a  pin 
fault  (TR3826) . 

Comprehension .  The  completeness  of  a  test  procedure/program  with  respect 
to  a  desired  level  of  fault  detection  and/or  fault  isolation.  Also,  the 
transparency  of  an  Automatic  Test  Generation  (ATG)  program  to  a  specific 
logic-implementing  element  or  primitive. 

Confidence  test.  A  go/no-go  test. 

Controllability.  An  attribute  of  equipment  design  which  defines  or  des¬ 
cribes  the  degree  of  test  control  which  may  be  exercised  at  internal  nodes 
of  interest  (TR3826) . 

CPU.  Central  Processing  Unit  (a  crtegory  of  hardware) . 

Critical  failure.  A  failure  which  results  in  a  casualty  (TR3826) . 

Critical  race.  In  digital  logic,  the  concurrent  change  of  two  or  more 
feedback  lines  which  may  result  in  any  one  of  two  or  more  stable  states 
being  entered  (TR3826) . 
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Deductive  simulation.  A  simulation  method  in  which  the  fed. lure -free  UUT 
is  first  simulated  and  its  responses  for  each  stimulus  confuted.  A 
deductive  program  is  then  used  to  analyze,  based  on  the  good  UUT  responses, 
what  failures  will  affect  the  status  of  which  primary  output. 

Delay  fault/delay  failure.  A  failure  in  a  digital  device  such  that  switch¬ 
ing  occurs  to  the  proper  level  but  does  so  outside  of  a  specified  time 
interval  (TR3826) . 

Dependent  fault/dependent  failure.  A  fault  which  is  caused  by  the  failure 
of  an  associated  item  (M1L-STD-721B) . 

Design  fault.  A  design  characteristic  of  either  hardware  or  software 
which  causes  or  materially  contributes  to  equipment  malfunction  indepen¬ 
dent  of  the  presence  of  hardware  failures  (TR3826) . 

Design  for  testability  (DFT) .  A  design  process  or  characteristic  thereof 
such  that  deliberate  effort  is  expended  to  assure  that  a  product  may  be 
thoroughly  tested  with  minimum  effort,  and  that  high  confidence  may  be 
ascribed  to  test  results  (TR3826) . 

Diagnostic  accuracy.  The  percentage  of  failures,  based  upon  the  failure 
population,  which  are  correctly  isolated  (TR3826) . 

Diagnostic  test.  A  test  designed  to  perform  fault  isolation. 

(TR3826) . 

Disruptive  test.  One  which  destroys  (changes)  the  state  of  the  hardware 
or  software. 

Don’t-care  state.  When  an  input  pattern  or  stimulus  is  created  for  a  UUT, 
it  is  very  common  that  only  a  portion  of  the  primary  inputs  will  be 
assigned  to  specific  values  (one  or  zero).  Those  which  are  not  specifi¬ 
cally  assigned  are  said  to  be  the  don't-cares. 

Dynamic  test.  A  test  of  one  or  more  of  the  signal  properties  or  rharac- 
teristics  of  the  equipment  or  of  any  of  its  constituent  items,  performed 
such  that  the  parameter(s)  being  observed  is  (are)  measured  and  assessed 
with  respect  to  a  specified  time  aperture  or  response  (ATG) . 


E 

Early  failures.  Also  birth  failures  or  burn-in  failures.  Failures  during 
the  early  period,  beginning  at  some  stated  time  and  during  which  the 
failure  rate  of  son^e  items  is  decreasing  rapidly  (IEEE) .  A  large  number 
of  internal  circuit  failures  probably  occur  in  this  period  due  to  manu¬ 
facturing  imperfections  or  design  errors  (TR3826) . 

End-to-end  run  time.  The  time  for  a  Test  Program  to  determine  that  a  UUT 
is  good  (MIL-STI>>2077  <AS)1, 

Equivalent  fault.  A  fault  X  is  equivalent  of  a  fault  Y  if,  and  only  if, 
a  system  containing  fault  X  has  the  same  observable  behavior  as  the  same 
system  containing  fault  Y  (JEEE/FTC) .  Two  or  more  faults ,  depending  on 
the  logic  structure,  which  create  the  same  response  for  a  complete  test 
set.  When  a  single  test  (one  or  several  stimuli)  is  applied  to  a  UUT, 
it  is  possible  that  two  or  more  faults  may  have  the  same  response,  but 
they  may  react  differently  for  another  test.  If  this  is  true,  the  faults 
are  not  equivalent. 

Error.  Any  discrepancy  between  a  computed,  observed,  or  measured 
quantity  and  the  true,  specified,  or  theoretically  correct  value  or 
condition  (IEEEI, 

Exact  match  fault  dictionary.  A  fault  dictionary  whose  successful  utili¬ 
zation  is  predicted  exclusively  upon  existence  of  exact  matches  of 
observed  fault  signatures  against  predicted  fault  signatures  enumerated 
in  the  dictionary  (TR3826) , 

External  ATE.  ATE  which  is  physically  separated  from  the  unit  under  test 
when  the  UUT  is  in  its  operational  environment  (TR3826) . 
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Fail-all  simulator.  Known  also  as  a  sequential  simulator;  all  faults 
simulated  one  at  a  time  in  serial  fashion. 

Failed  machine  response  (FMR) .  The  output  response  of  a  failed  UUT  when 
a  stimulus  is  applied. 

Fail  soft/soft  failure.  The  toleration  of  the  effects  of  a  predetermined 
number  of  failures  with  only  partial  loss  of  functional  capacity 
(IEEE/FTC) . 

Failure.  The  termination  of  the  ability  of  an  item  to  perform  its  re¬ 
quired  function  (IEEE) .  A  failure  is  the  functional  manifestation  of 
a  fault  (TR3826) .  A  malfunction  that  causes  degradation  or  complete 
loss  of  equipment  performance  (MIL-STD-1309B) .  The  inability  of  an  item 
to  perform  within  previously  specified  limits  (MIL-STD-721B) .  A  condition 
of  the  unit  under  test  which  causes  the  next  higher  assembly  to  perform 
in  an  unacceptable  manner  (I/JS) . 

Failure  analysis.  The  logical,  systematic  examination  of  an  item  or  ; 
diagram(s)  to  identify  and  analyze  the  probability,  causes,  and  const 
quences  of  potential  and  real  failures  (MIL-STD-721B  and  MIL-STD-1303B) . 

Failure  coverage.  An  attribute  of  a  test  expressed  as  the  percent  of 
failures  in  the  failure  population  which  that  test  will  detect  (TR3826,  . 

Failure,  dependent.  A  failure  which  is  caused  by  the  failure  of  an 
associated  item,  distinguished  from  independent  failure  (MIL-STD-1309i 
One  which  is  caused  by  the  failure  of  an  associated  item(s) .  Not  indepen¬ 
dent  (MIL-STD-721B)  . 

Fai lure ,  independent .  A  failure  which  occurs  without  being  related  to  t 

failure  of  associated  items,  distinguished  from  dependent  failure 
(MIL-STD-1309B) .  One  which  occurs  without  being  related  to  the  failure  of 
associated  items.  Not  dependent  (MIL-STD-721B) . 

Failure  mode.  A  failure  classification  (TR3826) . 

Failure  population.  The  failures  which  correspond  to  '■pecified  failure 
modes.  This  may  be  used  as  a  basis  for  the  design  md  evaluation  of 
tests . 

Failure,  random.  Any  failure  whose  occurrence  is  unpredictable  in  an 
absolute  sense  but  which  is  predictable  only  in  a  probabilistic  or 
statistical  sense  (MIL-STD-721B  and  MIL-STD-1309B) . 

Failure  rate.  The  number  of  failures  of  an  item  per  unit  measure  of  life 
(cycles,  time,  miles,  events,  etc.,  as  applicable  for  the  item) 
(MIL-STD-721B) . 
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Failure  resolution.  A  measure  of  the  capability  of  a  test  process  to 
perform  fault  (failure)  isolation  among  replaceable  units,  generally 
expressed  as  N  or  fewer  replaceable  units  XX%  of  the  time  (based  upon  the 
failure  population)  (TR3826) . 

Failure  universe/failure  population.  The  failures  which  correspond  to 
a  selected  fault  population.  This  is  used  as  a  basis  for  the  design  ad 
evaluation  of  tests  (TR3826) . 

Failure  universe.  There  are  many  ways  to  determine  the  failure  universe. 

A  user  may  specify  the  universe  that  suits  his  purpose.  When  an  ATG 
system  is  employed,  the  system  must  clearly  explain  how  the  universe  is 
defined,  and  ATG  statistics  must  be  based  on  this  specified  universe. 
Percent-detect  is  meaningless  unless  the  failure  universe  is  clearly 
defined.  Because  of  theoretical  as  well  as  practical  limitations,  the 
failure  universe  is  usually  defined  as  the  total  number  of  single 
stuck-at  failures  of  user  nodes.  Double  stuck-at  failures  and  bridge 
failures  are  sometimes  included,  but  they  have  to  be  specified.  When 
equivalent  circuits  are  used  by  an  ATG  the  ATG  node  failures  may  be  used 
as  the  universe  and,  in  this  case,  separate  statistics  should  be  given. 
Otherwise,  the  percent-detect  may  be  misleading  because  a  node  of  the 
equivalent  circuit  may  not  exist  in  the  actual  hardware  used  in  the  UUT. 

False  alarm.  An  indicated  fault  where  no  fault  exists.  (Does  not  include 
good  items  in  an  ambiguity  group.)  (MIL-STD-1309B) . 

False  alarm  rate.  The  frequency  of  occurrence  of  false  alarms  (TR3826) . 

False  reject.  An  item/device  incorrectly  identified  as  exhibiting 
faulty  performance  due  to  either  a  false  alarm  or  an  ambiguously 
isolated  fault. 

Fault .  A  physical  ''ondition  that  causes  a  device,  component,  or  element 
to  fail  to  perform  in  a  required  manner;  for  example,  a  short-circuit  or 
a  broken  wire  (IEEE).  A  degradation  in  performance  due  to  detuning, 
maladjustment,  misalignment,  failure  of  parts,  and  so  forth 
(MIL-STD-130DB) .  The  causative  failure  of  a  lower  level  assembly  within 
the  unit  under  test  (ultimately  a  fault  may  be  traced  to  a  physical  change 
of  a  component  of  the  system)  (I/JS) . 

Fault,  analog,  catastrophic.  A  defect  or  malfunction  in  an  analog 

component,  assembly,  or  system,  causing  a  sudden  change  in  its  operating 
characteristics  which  results  in  a  complete  lack  of  useful  performance  of 
the  device  or  system. 

Fault,  analog,  out-of-toler^ ace .  A  defect  or  malfunction  in  an  analog 
component,  assembly,  or  system,  in  which  a  performance  parameter 
approximates  but  falls  outside  the  prescribed  upper  limit  or  lower  limit 
for  that  parameter. 
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Fault  collapse.  Since  equivalent  faults  or  failures  cannot  be  distin¬ 
guished  without  manual  intervention,  it  is  desirable  to  have  a  preanalysis 
which  permits  equivalent  faults  to  be  classified  or  grouped  together. 

For  each  equivalent  class,  only  one  representative  can  be  used  for  both 
test  generation  and  fault  simulation;  the  efficiency  of  the  AT  system  is 
thus  enhanced.  Since  the  members  of  all  equivalent  classes  axe  shown  in 
the  fault  dictionarv,  the  information  is  complete  and  the  resolution  is 
the  same. 

Fault  coverage.  An  attribute  of  a  test  or  test  procedure  expressed  as 
the  percent  of  faults  of  the  total  fault  population  which  that  test  or 
test  procedure  will  detect  (TR3826) . 

Fault  detection.  A  process  which  discovers  or  is  designed  to  discover  the 
existence  of  faults;  the  act  of  discovering  existence  of  a  fau-t  (TR3826) . 
One  or  more  tests  performed  to  determine  if  any  malfunctions  or  faults  are 
present  in  a  unit  (MIL-STD-1309B) . 

Fault  dictionary.  A  list  or  elements  where  each  element  consists  of  a 
test  and  all  the  faults  detected  by  that  test  (IEEE/FTC) .  often  only  the 
LRUs  which  contain  the  faults  are  listed  (TR3826) .  A  data  set  or  file 
created  by  an  ATG  system.  It  relates  the  test  stimulus,  the  responses 
of  the  failure-free  machine,  the  various  failed-machine  responses,  the 
nature  of  the  failure,  and  the  associated  replaceable  units.  The  fault 
dictionary  may  only  show  the  failure-free  UUT  responses  and  indicate 
which  stimuli  (pattern)  cam  detect  what  failure (s)  from  which  primary 
output  (pin)  and  the  associated  replaceable  units.  The  difference  in  • 
data  structure  results  from  the  different  simulation  techniques  used. 

Fault,  digital,  open.  A  defect  or  malfunction  in  a  logic  circuit  or 
digital  device,  resulting  from  a  discontinuity  (break)  in  a  signal  pat.i 
such  that  the  output  response  for  any  input  stimuli  becomes  indeterminate. 

Fault,  digital,  stuck-at-one .  A  defect  or  malfunction  in  a  logic  element, 
circuit,  or  digital  device,  such  that  one  or  more  of  its  outputs  takes  on 
the  value  of  a  logical  1  and  cannot  he  changed  from  this  high  state 
regardless  of  the  input  stimuli. 

Fault,  digital,  stuck-at-zero.  A  defect  or  malfunction  in  a  logic  element, 
circuit,  or  digital  device,  such  that  one  or  more  of  its  outputs  takes 
on  the  value  of  a  logical  0  and  cannot  be  changed  from  this  low  state 
regardless  of  the  input  stimuli. 

Fault  dominance.  a  fault  X  dominates  a  fault  Y  if,  and  only  if,  eve^y 
test  for  Y  is  a  test  for  X,  where  X  and  Y  ar_>  two  faulis  which  may  occur 
in  a  svstem  (IEEE/FTC) . 
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Fault  indicator.  A  device  which  presents  a  visual  display,  audible  alarm, 
and  so  fprth,  when  a  failure  or  marginal  condition  exists  (MIL-STD-1309B) . 

Fault  isolation.  Where  a  fault  is  known  to  exist,  a  process  which  identi¬ 
ties  or  is  designed  to  identify  the  location  of  that  fault  within  a  small 
number  of  replaceable  units  (TR3826) .  Tests  performed  to  isolate  faults 
within  the  unit  under  test  (MIL-STD-13Q9B) . 

'Fault  latency  time.  The  duration  of  time  during  which  an  existing  fault 
is  undetected;  the  elapsed  time  between  fault  occurrence  and  fault 
detection. 

Fault  localization.  Where  a  fault  is  known  to  exist,  a  process  which 
identifies  or  is  designed  to  identify  the  location  of  that  fault  within 
a  general  area  of  equipment.  Fault  localization  may  be  less  specific 
than  fault  isolation  (TR3826) . 

Fault  masking.  A  fault  X  masks  a  fault  Y  if  no  test  for  fault  Y  is  a 
test  for  the  faults  X  and  Y  occurring  jointly  in  a  system  (IEEE/FTC) . 

Fault  population.  The  totality  cf  faults  which  may  be  incurred  by  a 
device  (TR3826) . 

Fault  prediction.  A  process  used  to  r-redict  that  some  component  will  be 
out  of  tolerance  before  the  next  scheduled  maintenance  period  based  upon 
the  present  measurement  of  component  parameters  (TR3826) . 

Fault  resolution.  A  measure  of  the  capability  of  a  test  process  to 

perform  failure  isolation  among  replaceable  units,  generally  expressed  as 
"n  or  fewer  replaceable  units  XX%  of  the  time"  (based  upon  the  faul4- 
population)  (TR3826) .  A  measure  of  the  capability  of  an  ATG  to  perform 
failure  isolation.  In  this  case,  identification  of  the  failed  replaceable 
units  (RU’s)  rather  than  the  failures  serves  as  a  measure  of  the  resolu¬ 
tion  achieved.  The  fault  resolution  is  considered  satisfactory  if,  after 
application  of  a  test  set  to  a  UUT,  the  number  of  RU’s  which  are  identi¬ 
fied  as  possibly  contributing  to  a  detected  fault  is  equal  to  or  less 
than  a  specified  number. 

Fault  signature.  An  output  test  vector  resulting  from  the  testing  of  a 
unit  containing  one  or  more  faults  (TR3826) . 

Fault  simulation.  A  process  which  admits  prediction  or  observation  of 
system  behavior  in  the  presence  of  a  specified  fault  without  infliction 
of  that  fault  upon  that  system.  The  process  demands  modeling  of  either 
the  fault,  the  system  or  both  (TR3826) . 

Fault  symptom.  A  measurable  or  visible  abnormality  in  an  equipment 
parameter  (MIL-STD-1309B) . 
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Fault  'tolerance.  The  capacity  of  a  computer ,  subsystem,  or  program  to 
withstand  the  effects  of  internal  faults;  the  number  of  error-producing 
faults  a  computer,  subsystem  or  program  can  endure  before  normal 
functional  capability  is  impared  (IEEE/FTC) . 

FDI.  Fault  Detection  Isolation  system.  Generic  term  encompassing  all  of 
the  above  terms. 


Field  failures.  In-service  failures ,  characterized  by  an  absence  of 
burn-in  failures  (TR3826) . 

FIT.  Fault  Isolation  Test.  Iterative  step  of  BIT  that  isolates 

failure  to  a  single  component  or  narrows  failure  to  a  group  of  components . 

Functional  fault.  A  fault  which  can  be  described  by  a  change  in  function 
of  some  identifiable  portion  of  a  system  (IEEE/FTC).  A  failure. 

Functional  model.  When  a  circuit  can  be  described  as  a  functional  unit, 
it  xs  not  necessary  to  show  its  logic  structure.  Instead,  a  function 
name  or  code  is  given.  A  UUT  can  thus  be  expressed  by  a  functional  model 
which  is  a  network  containing  several  function  blocks. 

Functional  modularity.  The  splitting  of  a  system  into  parts  cr  modules 
based  on  the  function  or  purpose  of  those  parts  (IEEE/FTC) . 

Functional  test.  A  test  which  is  intended  to  exercise  an  identifiable 
function  of  a  system  (IEEE/FTC) .  The  function  is  often  tested  indepen¬ 
dent  of  the  hardware  implementation  of  the  function  (TR3826) . 

Functional  partitioning.  The  physical  or  electrical  separation  of  system 
elements  along  interfaces  which  define  and  isolate  these  elements  on 
bases  of  function  or  purpose  (TR3826) . 
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Gate- level  model.  A  modeling  technique  in  which  equivalents  for  high- 
level  logic  elements  are  constructed  from  basic  gating  elements. 

Global-initialization  algorithm.  An  algorithm  is  a  systematic  procedure 
that  can  be  programmed  on  a  digital  computer,  which  will  terminate 
either  when  the  operation  is  complete  or  when  it  has  reached  its  theo¬ 
retical  limit.  For  initialization  of  sequential  circuits  in  ATG  systems, 
the  algorithm  must  not  depend  on  the  outputs  of  the  memory  elements  to 
be  initialized.  It  should  "compute"  a  sequence  of  input  patterns  so 
that  when  they  are  applied  to  the  UUT,  the  unknown  states  become  known 
and  unique.  A  global- initialization  algorithm  is  one  which  is  capable 
of  initializing  all  memory  elements  of  a  UUT  to  its  limit. 

Go- chain  test.  A  test  or  a  group  of  tests  which  evaluate  a  UUT  performance 
function  or  parameter. 

Good  machine  response  (GMR) .  The  output  response  of  a  failure- free  UUT 
when  a  stimulus  is  applied. 

Go/no-go  test.  A  test  designed  to  yield  a  "test  pass"  or  "go"  indication 
in  the  absence  of  faults  in  a  UUT,  and  a  "test  fail"  or  "no-go"  indi¬ 
cation  in  the  presence  of  fault(s)  (TR-3826). 

Go /no-go.  A  set  of  terms  (in  colloquial  useage)  referring  to  the  condition 
or  state  of  operability  of  a  unit  which  can  only  have  two  parameters: 

(a)  go,  functioning  properly,  or  (b)  no-go,  not  functioning  properly 
(MIL-STI>-  1309B)  . 
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Hard  core.  That  kernel  of  circuitry  in  a  processor  or  system  which  must 
be  functioning  properly  in  order  for  that  processor  or  system  to  success¬ 
fully  execute  tests  of  other  portions  of  itself  (TR-3826). 

Hard  core  failure.  A  failure  in  the  hard  core  logic  of  a  system  which 
inhibits  normal  self- test  of  the  system  (TR-3826) . 

Hard  detect.  If  the  responses  of  a  failed  UUT  and  a  good  UUT  differ  by  at 
least  one  bit,  the  failure  can  be  definitely  detected  and  is  called  a 
hard  detect. 

Hard  fault.  A  fault  which  has  effectively  reached  the  limit  of  its  effect 
upon  the  performance  of  the  next  higher  assembly  (I/JS) . 

Hazard.  In  combinational  logic,  the  possible  transient  changing  of  an 
output  due  to  internal  delay  characteristics.  A  hazard  is  harmful  if  it 
affects  the  states  of  memory  devices. 

Hazards  -  static  hazard.  Is  a  transition  between  a  pair  of  adjacent  input 
states  which  both  produce  the  same  output,  during  which  transition  an 
opposite  output  momentarily  occurs.  Dynamic  Hazard  is  a  transition  between 
a  pair  of  adjacent  input  states — one  of  which  produces  a  "1"  output  and 
the  other  of  which  a  "0"  output — during  which  transition  it  is  possible 
for  both  a  momentary  "0"  output  and  a  momentary  "1"  output  to  occur. 

HW.  An  abbreviation  for  hardware. 
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Idling .  When  resource  is  not  being  used  by  the  system  but  is  scheduled 
to  be  used. 

Impossible  detect.  Failures  which  cannot  be  detected  by  any  test  (ATG) . 

Independent  fault/independent  failure.  A  fault  which  occurs  without  being 
related  to  the  failure  of  associated  items  (MIL-STD-721B) . 

Initialize.  (1)  To  establish  an  initial  condition  or  starting  state; 
for  example,  to  set  logic  elements  in  a  digital  circuit  or  the  contents 
of  a  storage  location  to  a  known  state  so  that  subsequent  application  of 
digital  test  patterns  will  drive  the  logic  elements  to  another  known 
state;  and  (2)  to  set  counters,  switches,  and  addresses  to  zero  or  other 
starting  values  at  the  beginning  of,  or  at  prescribed  points  in,  a 
computer  routine  (MIL-STD-1309B) . 

Input  test  vector.  A  test  pattern  (TR3826) . 

Interface  adapter.  A  device  designed  to  provide  a  compatible  connection 
between  the  unit  under  test  and  the  test  equipment.  It  may  include 
proper  stimuli  or  loads  not  contained  in  the  test  equipment. 

Interface  device  (ID) .  The  ID  shall  provide  me chemical  and  electrical 
connection  and  signal  conditioning,  if  required,  between  the  ATE  and  the 
UUT  (MIL-STD-2077 (AS) ) . 

Interference  testing.  A  type  of  on-line  testing  that  requires  disruption 
of  the  normal  operation  of  the  unit  under  test  (see  non-interference 
testing)  (MIL-STD-13093) .  Off-line  testing  (TR3826) . 

Iterative  tesc.  A  test  which  must  be  repeated  several  times  with  new 
values  for  some  of  the  test  parameters  each  time,  provided  that  all  re¬ 
quired  new  test  parameter  values  can  be  obtained  as  a  function  of  the 
iteration  number  or  .  s  a  unique  transformation  on  the  values  used  in  the 
immediate  preceding  iteration.  The  test  to  be  iterated  is  defined  as  a 
unique  test  and  the  number  of  iterations  is  the  number  of  additional 
iterative  tests. 

Intermittent  fault.  A  temporary  fault  (IEEE) . 

Inverted  -  pyramid/building  -  block.  Descriptive  terms  characterizing  a 
test  or  test  technique  whereby  the  smallest  possible  portions  of  hardware 
are  tested  first  in  the  test  sequence,  and  subsequent  tests  utilize 
previously  verified  hardware  for  execution  (TR3826) . 
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Latent  fault  time.  The  extent  or  duration  of  time  during  which  an  existing 
fault  is  undetected;  the  elapsed  time  between  fault  occurrence  and  fault 
detection  fTR-3826) . 

Learn- mode  testers.  The  utilization  of  random  test  patterns  as  stimuli 
for  a  circuit  to  produce  a  change  in  state  at  the  output.  If  no  change 
in  state  occurs  with  a  given  pattern,  that  pattern  is  discarded.  This 
process  is  continued  until  a  series  of  active  patterns  are  put  together 
to  form  a  test.  This  type  of  test  generation  cannot  perform  fault  iso¬ 
lation. 

Line  replaceable  unit  (LRU.  A  unit  which  is  designated  by  the  plan  for 
maintenance  to  be  removed  upon  failure  from  a  larger  environment 
(MIL-STD1309B) . 

LRU.  A  generic  term  that  may  be  defined  in  terms  of  both  Avionic  equipment 
and  Ground  Systems  equipment.  For  Avionic  equipment  or  systems,  it  is 
Line  Replaceable  Unit.  For  Ground  based  equipment  or  systems,  it  is 
Lowest  Replaceable  Unit.  LRU  is  defined  as  a  unit  which  is  designated 
by  the  plan  for  maintenance  to  be  removed  upon  failure  from  a  larger 
entity  (equipment,  system)  in  the  latter's  operational  environment 
(MIL-STD- 1309B) .  A  LRU  is  composed  entirely  of  SRU’s. 

LSA.  Logistic  support  analysis.  A  systematic  process  for  defining  logis¬ 
tic  support  requirements,  generally  in  conformance  with  M1L-STD-1388. 
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Macro  block.  In  some  ATG  systems,  the  users  can  define  a  logic  configu¬ 
ration  as  a  single  macro.  This  is  convenient  if  the  logic  configuration 
is  used  repeatedly  many  times  in  a  UUT.  Moreover,  the  chance  of  making 
errors  is  greatly  reduced  when  macro  bl ocks  cure  used.  Since  the  ATG 
system  will  expand  the  macro  blocks  in  terms  of  primitives,  there  is  no 
limitation  on  how  simple  or  complex  a  macro  block  can  be.  Also,  there 
is  no  need  to  write  a  special  interpretive  subroutine  Tor  each  macro 
block.  Macro  blocks  are  similar  to  macro  instructions  for  assembly 
languages.  They  are  also  referred  to  as  "open  macros"  because  they  are 
expanded  by  the  ATG  system. 

Macro  function  (functional  macro) .  The  functional  macro  is  a  "closed 
macro"  —  i.e. ,  one  not  expanded  by  the  ATG  system.  The  functional 
macro  is  represented  as  a  "single  block"  —  i.e.,  RAM,  ROM,  counters, 
shift  registers,  etc.  This  single-block  representation  saves  memory 
space.  Special  sub- routines  are  used  to  corpute  the  input/output  rela- 
tionaships.  In  addition  to  space  savings,  this  approach  can  enhance 
test-generation  performance.  The  functional  macro  is  generally  used  for 
MSI  and  LSI,  where  fault  resolution  is  relatively  unimportant  (cf.  func¬ 
tional  model) . 

Maintainability.  A  characteristic  of  equipment  design  and  installation 
which  is  expressed  as  the  probability  that  an  item  will  be  retained  in 
or  restored  to  a  specified  condition  within  a  given  period  of  time,  when 
the  maintenance  is  performed  in  accordance  with  prescribed  procedures  and 
resources  (MIL-STD-721B) . 

Maintenance  replaceable  unit.  An  LRU  (TR-3826) . 

Malfunction.  An  error  (TR-3826). 

Marginal  fault.  A  failure  such  that  some  equipment  function  is  impaired 
or  out  of  tolerance  and  is  of  a  nature  such  that  catastrophic  failure 
does  not  occur  (TR-3826). 

Mean- time- to- repair  (MTTR) .  The  total  corrective  maintenance  time  divided 
by  the  total  number  of  corrective  maintenance  actions  during  a  given 
period  of  time  (MIL-STD- 721B) .  (Includes  actions  due  to  false  alarms.) 

Mean  time  betweeh  ma  '.ntenance  (MTBM)  .  The  mean  of  the  distribution  of  time 
intervals  between  maintenance  action  (either  preventive,  corrective  or 
both)  (TR-3826). 
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Mean  time  to  isolate.  The  average  time  required  to  achieve  fault  isolation 
as  measured  from  the  time  of  fault  detection  to  the  time  of  fault  iso¬ 
lation  (TR-3826). 

Micro  functional.  Cf.  primitive  model. 

Mistake.  A  human  action  that  produces  an  unintended  result  (IEEE). 

Module.  A  pluggable  board  (card)  upon  which  circuit  components  are 
mounted. 

Multiple  failure.  A  joint  occurrence  of  two  or  more  single  failures  (IEEE). 
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Net.  The  inputs  and  outputs  of  logic  gates  are  usually  referred  to  as 
nodes.  A  net  is  a  group  of  nodes  connected  together. 

Non-interference  testing.  On-line  testing  (TR-3826) .  A  type  of  on  line 
testing  that  may  be  carried  out  during  normal  operation  of  the  unit  under 
test  without  affecting  the  operation  (see  interference  testing 
( MI L-STD- 1309B) . 
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Observability.  An  attribute  of  equipment  design  which  describes  the  extent 
to  which  signals  of  interest  may  be  observed  (TR-3826) . 

Off-line.  (1)  Operation  of  input/output  and  other  devices  not  under 
direct  control  of  a  device;  and  (2)  Peripheral  equipment  operated  out¬ 
side  of,  and  not  under  control  of  the  system,  for  example,  the  off-line 
printer  (MIL-STD-1309B) . 

Off-line  ATE.  The  testing  of  a  unit  Under  test  with  the  unit  removed  from 
its  normal  operational  environment. 

Off-line  testing.  Testing  of  the  unit  under  test  removed  from  its  opera¬ 
tional  environment  or  its  operational  equipment.  Shop  testing 
(MIL- STD- 1309B) .  Test  of  a  unit  under  test  (UUT)  with  the  unit  removed 
from  its  normal  operational  environment  (ATG) . 

Off-line  test  equipment.  Equipment  used  to  perform  tests  on  a  UUT  with 
the  unit  removed  from  its  normal  operating  environment  (ATG) . 

On-line.  Operation  of  an  input/output  device  as  a  unit  of  the  system  under 
programmed  control  of  the  system  (MIL-STD-1309B) . 

On-line  ATE.  The  testing  of  a  unit  under  test  with  automatic  test  equip¬ 
ment  while  in  its  operational  environment. 

On-line  test.  Test  of  a  UUT  in  its  operational  environment  (MXL-STD-1309B) . 

On-line  testing.  Testing  of  the  unit  under  test  in  its  operational  environ¬ 
ment  (see  interference  testing  and  non-interference  testing  (MIL-STD-1309B). 

On-line  test  equipment.  Equipment  used  to  perform  tests  on  a  UUT  while 
the  unit  is  in  its  normal  operating  environment  (ATG) . 

Open  fault.  A  fault  caused  by  an  electrical  separation  of  normally  elec¬ 
trically  connected  points  ( IEEE/FTC) . 

Output  test  vector.  An  ordered  set  of  simultaneous ly  observed  output 
values  (TR-3826). 
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Parallel  simulation.  For  digital  UUT's,  all  operations  are  bit- independent. 
Since  a  word  of  a  computer  usually  consists  of  many  bits,  several  failures 
can  be  simulated  simultaneously  through  a  failure-injection  mechanism. 

This  is  the  most  popular  and  accurate  method  of  simulation. 

Parametric  fault.  A  fault  vAiich  causes  some  parameter  for  a  device  to 
have  a  value  outside  its  specified  range  (IEEE/FTC) . 

Parametric  test.  The  measure  of  circuit  characteristics  to  ascertain  that 
they  fall  within  specified  tolerances  (ATG) . 

Pattern  sensitive  failure.  A  component  failure,  usually  internal  to  the 
component,  whose  effect  at  the  component’s  output  pin(s)  is  dependent 
upo  the  input  applied  (TR-3826) . 

Passive  test.  Non-active  testing  (TR-3826). 

Percent  detect.  Gross  percent- detect  of  user  nodes  is  the  total  number  of 
user-node  failures  detected — including  possible  detects — divided  by  the 
failure  universe  based  on  user  nodes.  Gross  percent-detect  of  ATG  nodes 
is  the  total  number  of  failures  detected — including  possible  detects — 
divided  by  the  failure  universe  based  on  ATG  nodes.  Net  percent-detect 
is  the  total  number  of  failures  detected  positively — not  including 
possible  detects — divided  by  the  defined  failure  universe.  Adjusted  net 
percent-detect  is  the  total  number  of  failures  detected  positively  divi¬ 
ded  by  the  defined  failure  universe,  which  does  not  include  impossible 
detects . 

PIP.  Peripheral  interface  device.  (A  hardware  category.) 

Pin  fault.  A  fault  which  is  present  at  a  single  input  or  output  pin  of  a 
component  or  module  (TR-3826). 

Possible  detect.  When  a  hazard- free  UUT  is  under  failure  condition  or 
when  a  UUT  is  designed  with  clock  pulses  and  is  tested  statically,  races, 
hazards,  and  even  oscillations  can  occur.  Consequently,  the  output 
response  may  contain  "indeterminate"  states.  Also,  a  UUT  may  not  be 
completely  initialized,  causing  the  output  response  to  contain  "unknown" 
states.  In  both  cases,  if  the  unknown  or  indeterminate  bits  of  a  failed 
UUT  response  are  its  only  differences  from  the  good  UUT  response,  it  is 
called  a  possible  detect.  For  binary  logic  the  unknowns  or  indeterminates 
can  only  be  either  "1"  or  "0",  hence  the  failure  is  sometimes  detectable. 
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An  independent  fault  (TR-3826) . 


Primitives .  The  basic  blocks  or  operators  used  by  an  ATG  system.  There 
are  several  levels  of  primitives  used  by  different  ATG  systems.  Some 
systems  vise  combinational  gates  or  Boolean  operators  only.  Other  systems 
include  sequential  elements  such  as  latches,  flip-flops,  delay  lines, 
and  monostables  in  their  primitive  sets.  Some  systems  consider  counters, 
shift  registers,  ROM,  and  RAM  as  primitives.  Primitives  usually  are 
expanded  into  equivalent  circuits  but  are  handled  as  single  blocks  by 
interpretive  subroutines. 

Primitive  model.  When  a  UUT  is  described  by  primitives,  the  result  is  a 
primitive  model.  It  is  actually  a  micro-functional  model,  which  is  one 
level  higher  than  the  gate-level  model. 
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Races.  When  a  sequential  cirt  .it  goes  from  one  state  to  another  because 
of  an  input  change/  the  input  stimulus  will  usually  cause  the  internal 
variables  to  change.  During  this  time,  transient  conditions  can  be 
produced  because  of  the  proj  agation  and  switching  delays  of  the  com¬ 
ponents.  The  timing  relationships  which  occur  among  the  internal  vari¬ 
ables  during  the  transition  sequence  are  affected  by  these  transients  and 
are  termed  races.  Because  sequential  circuits  have  memory  elements, 
these  races  may  cause  the  state  transition  of  the  circuit  to  go  wrong 
momentarily  or  permanently.  A  race  that  can  cause  a  sequential  circuit 
to  go  to  a  wrong  state  permanently  is  called  a  critical  race  and  must  be 
avoided. 

Random  fault/random  failure.  An  intermittent  fault  whose  occurrence  is 
predictable  only  in  a  statistical  sense. 

Readiness.  A  state  of  being  ready  to  successfully  perform  or  being  in  the 
act  of  successfully  performing  a  defined  mission  (TR-3626) . 

Readiness  test.  A  test  specifically  designed  to  determine  whether  an 

equipment  or  system  is  operationally  suitable  for  a  mission  (MIL-STD-1309B). 

Real-time  test  A  test  of  one  or  more  of  the  signal  properties  or  charac¬ 
teristics  of  an  equipment  or  any  of  its  constituent  items,  performed  such 
that  the  parameter (s)  being  observed  is  (are)  measured  and  assessed  while 
the  equipment  is  operating  at  its  normal  frequency  or  timing. 

Reconf ig ur ation .  A  repair  strategy  in  which  failing  components  are  switched 
out  of  operation  and  replaced  1  failure- free  components  (1EEE/PTC) . 

Recovery.  The  continuation  of  system  operation  with  error- free  data  after 
an  error  occurs  (IEEE/FTC) . 

Redundant redundancy.  The  introduction  of  auxiliary  elements  and  com¬ 
ponents  into  a  system  to  perform  the  same  function  as  other  elements  in 
the  system  for  the  purp'  se  of  improving  reliability  and  safety  (IEEE) . 

Also,  the  use  of  additional  components,  programs  or  repeated  operations, 
not  normally  required  by  the  system  to  execute  its  specified  tasks,  to 
overcome  the  effects  of  failures  (IEEE/FTC). 

Redundant  failure.  A  failure  whose  occurrence  in  a  system  do._s  not  ter¬ 
minate  system  ability  to  perform  any  required  function  (IEEE/FTC) . 

Repeatability.  A  test  characteristic  uch  that  repeated  application  of  a 
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given  set  of  stimuli  to  a  (JUT  yields  identical  results  (TR-3826) . 
Response.  The  observable  reaction  of  a  device  to  stimulus  (TR-3826) . 
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Secondary  failure.  One  or  more  dependent  faults  (TR3826) .  Those  failures 
which  occur  as  a  result  of  a  previous  malfunction  depicted  as  the  primary 
failure. 

Self-test.  Built-in  test  (TR3826) .  A  test  or  series  of  tests,  performed 
by  a  device  upon  itself,  which  shows  '-'hether  or  not  it  is  operating 
within  designed  limits.  This  includt-o  test  programs  on  computers 
and  automatic  test  equipment  which  check  out  their  performance  status 
and  readiness  (MIL-STD-1309B) . 

Self-test  capability.  The  ability  of  a  device  to  check  its  own  circuitry 
and  operation.  The  degree  of  self rtest  is  dependent  on  the  ability  to 
fault  detect  and  isolate  (MIL-STD-1309B) . 

Shop  non-ambiguity  ratio.  The  shop  non-ambiguity  ratio  is  the  ratio  of 
the  number  of  detected  faults  which  can  be  isolated  with  certainty  to 
the  total  number  of  detected  faults.  A  shop  non-ambiguity  ratio  of  1.0 
is  of  course  a  design  goal,  but  not  usually  to  be  economically  attained. 

Short  fault.  A  fault  caused  by  an  electrical  connection  between  normally 
electrically  separated  points  (IEEE) . 

SIT  (System  Integration  Test) .  Built-in  system  test  that  is  centrally 
integrated,  i.e.,  BIT  data  is  analyzed  through  a  central  computer  before 
Fault  Detection/Isolation  can  be  determined. 

Soft  fault.  A  fault  which  has  not  reached  the  limit  of  its  effect  upon 
the  performance  of  the  next  higher  assembly  (I/JS) . 

Solid  fault.  A  permanent  fault  (IEEE) . 

Specified  fault  population.  A  subset  of  the  fault  population  which  is 
used  as  the  basis  for  defining  the  failure  universe  (TR3826) . 

SRU  (Shop  Replaceable  Unit) .  A  generic  term  which  includes  all  the 
packages  within  a  LRU,  which  may  include  circuit  boards,  chassis, 
wiring  harnesses  and  piece  parts  removed  at  the  shop  (intermediate  or 
depot)  level. 

Standby  redundance.  That  redundance  wherein  the  alternative  means  of 
performing  the  function  is  inoperative  until  needed  and  is  switched 
in  upon  failure  of  the  primary  means  of  performing  the  function  (IEEE). 
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Static  functional  test.  A  test  in  which  measurement  is  made  of  the 
outputs  df  a  unit  under  test  tUUTl  after,  and  only  after,  these  outputs 
have  stabilized  with  respect  to  a  given  input  stimulus.  Further,  the 
measurement  and  assessment  is  made  only  with  respect  to  the  specific, 
overall  action  or  purpose  which  the  UUT  is  intended  to  perform  or  serve. 

Static  test.  A  test  in  which  measurement  is  made  on  a  UUT  after,  and 
only  after,  these  outputs  have  stabilized  with  respect  to  a  given  input 
stimulus  (ATG  modified) . 

Stimulus .  Any  physical  or  electrical  input  applied  to  a  device  intended 
to  produce  a  measurable  response  (M1L-STD-1309B) . 

Stuck  fault/stuck  failure.  A  failure  in  which  a  digital  signal  is 
permanently  held  in  one  of  its  binary  states  (IEEE) . 

Supplementary  data.  Supplementary  data  consists  of  information,  text, 
schematics  and  logic  diagrams  necessary  for  analysis  of  the  TPS  and  UUT 
in  event  of  a  problem  or  anomaly  during  the  testing  process.  The  amount 
and  content  of  the  supplementary  data  is  contingent  upon  the  capability 
of  the  ATE  to  store  and  display  required  information  automatically 
(MIL-STD-2077 (AS) ) . 

SW.  An  abbreviation  for  software. 

Symptom.  The  manifestation  of  evidence  of  a  particular  failure  condition 
(TR3826) . 
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Ternary  simulator.  A  program  which  establishes  a  representation  of  a  logic 
circuit  or  configuration  based  upon  a  computer-directed  and/or  processed 
model  of  the  logic  circuit  or  configuration.  Node  points  and  output 
pins  of  the  simulated  circuit/configuration  are  permitted  to  take  on  three 
values — logical  1,  logical  0,  or  X  (unknown  state) — in  sequences  and 
along  paths  in  accord  with  program  rules,  in  order  to  derive  fault-detec¬ 
tion  and  fault-isolation  information  for  the  logic  circuit  or  configuration 
represented. 

Test.  A  procedure  or  action  taken  to  determine  under  real  or  simulated 
conditions  the  capabilities,  limitations,  characteristics,  effectiveness, 
reliability,  or  suitability  of  a  material,  device,  system,  or  method 
(MIL-STD-1309B) .  A  segment  of  a  source  program  which  contains  as  a 
minimum,  a  comparison  between  a  measured  value  and  program  defined  limits, 
as  well  as  the  branching  instructions  directing  the  program  to  proceed 
based  on  the  results  of  the  comparison.  Generally,  it  also  will  contain 
source  statements  to  apply  stimuli  and  to  set  up  and  make  measurements. 

When  the  stimuli  being  applied  must  be  calibrated,  as  when  incrementing, 
or  as  when  RF  amplitude  calibration  is  required,  each  comparison  in  the 
incrementing/calibration  routine  is  counted  as  a  test. 

Testability.  A  design  characteristic  which  allows  the  status  (operable, 
inoperable,  or  degraded)  of  a  system  or  any  of  its  subsystems  to  be 
confidently  determined  in  a  timely  fashion  (TR-3826) .  Testability 
attempts  to  quantify  those  attributes  of  avionic  system  designs  which 
facilitate  detection  and  isolation  of  faults  that  affect  system  per¬ 
formance.  Testability  has  been  defined  as  ".the  characteristic  of  a 
design  which  allows  the  status  of  a  system  or  any  of  its  subsystems  to 
be  confidently  determined  in  a  timely  fashion. "  This  definition  should 
be  expanded  to  include  the  concept  of  isolating  and  repairing  detected 
faults  in  a  confident  and  timely  fashion  so  as  to  minimize  repair  time. 
Testability  is  the  unambiguous  isolation  of  a  fault  to  an  appropriate 
level  of  replaceable  equipment  such  that  the  proper  maintenance  is 
implemented  with  a  minimum  expenditure  of  time  and  resources.  Speci¬ 
fication  of  that  appropriate  level  is  a  most  important  consideration, 
which  must  be  addressed  early  in  the  design  cycle.  (Subtask  A- 3  Design 
for  Testability,  Final  Report.  U.  S.  Government  Printing  Office,  1977, 
727-156/1-3,  p  35.) 

Testability  reflects  the  susceptibility  of  a  PCB  to  the  detection  of 
all  faults,  to  rapid  and  accurate  isolation  to  the  faulty  component 
without  ambiguity  and  to  functional  test  and  thereby  verification  of 
PCB  performance  as  specified  and/or  required.  (Testability  Investi- 
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gation  Attachment  IV  to  Interim  Report,  Manufacturing  Methods  and 
Technology  for  Digital  Fault  Isolation  for  Printed  Circuit  Boards, 
Contract  Mo.  DAAK40-78-C-0290. ) 

Testability,  comprehensive.  An  overall  testability  design  characteristic 
which  includes  both  hardware  design  and  test  design. 

Testability,  inherent.  A  hardware  testability  design  characteristic  which 
does  not  include  consideration  of  test  stimulus/response  data. 

Test  accuracy  ratio  (TAR) .  The  ratio  of  the  measurement  uncertainty  of 
the  UUT  to  the  measurement  uncertainty  of  the  ATE.  For  example,  if  it 
is  required  that  a  UUT  output  be  accurate  to  5%  and  the  ATE  accuracy 
in  measuring  the  parameter  is  0.001%,  then  the  TAR  is  5000 
(MIL-STD-2077 (AS) ) . 

Test  effectiveness.  A  measure  which  reflects  the  fault  coverage  and 
fault  resolution  provided  by  a  test  (TR-3826). 

Test  generation.  The  process  designing  tests  or  test  stimuli  (TR-3826). 

Test  length.  The  number  of  tests  in  a  test  sequence  (TR-3826). 

Test  pattern.  A  simultaneous  or  parallel  definition  of  all  the  inputs  of 
a  system  (IEEE/FTC) . 

Test  point.  A  node  within  a  circuit  or  system  which  can  be  measured  or 
stimulated  to  facilitate  testing  (TR-3826) . 

Test  program  (TP) .  The  TP  contains  a  coded  sequence  which,  when  executed 
by  the  ATE,  will  provide  the  system  a  set  of  instructions  sufficient  to 
accomplish  the  objective  of  the  TP.  The  objective  of  the  TP  is  to 
automatically  ascertain  the  operational  readiness  condition  of  the  UUT. 
The  TP  isolates  faults  to  levels  as  defined  in  AR-10  (MIL-STD-2077 (AS) ) . 

Test  program  instruction  (TPI) .  The  TPI  provides  information  needed  for 
testing,  (E.G. ,  hook-up,  probe  point  locations  or  other  programmed 
operator  intervention)  which  cannot  be  conveniently  provided  by  the  ATE 
under  control  of  the  TP.  Appropriate  contents  in  large  part,  are 
dependent  on  the  ATE  being  used.  For  example,  an  ATE  with  a  sophisti¬ 
cated  display  subsystem  could  provide  diagram  and  test  conveniently  and 
quickly  and  wouJ4  need  minimal  TPIs  while  on  ATE  with  a  slow  speed, 
test  oriented,  output  device  would  need  detailed  and  extensive  TPIs 
(MIL-STD-2077 (AS)). 

Test  program  set.  The  TPS  consists  of  those  peculiar  items  necessary  to 
test  a  UUT  on  an  ATE.  This  includes  the  electrical,  mechanical, 


instructional  and  logical  decision  elements  (MIL-STD-2077(AS) ) . 

Test  program  set  integration.  The  process  of  debugging  the  Test  Program, 
Interface  Device  and  Test  Program  Instruction  (MIL-STD-2077(AS) ) . 

Test  sequence.  A  specific  order  of  related  tests  (MIL-STD-1309B) . 

Test  validation.  Actions  taken  to  determine  if  test  responses  for  a 
fault-free  UUT  are  in  agreement  with  desired  values  (TR-3826) . 

Test  verification.  Actions  taken  to  assure  that  a  test  meets  specifi¬ 
cation  of  fault  coverage  and  fault  resolution  (TR-3826) . 

Topology.  When  applied  to  a  circuit,  topology  means  the  structure  of  the 
circuit. 

Transient  failure.  A  failure  induced  by  a  momentary  or  temporary  external 
factor  such  as  input  power  fluctuation,  excessive  ambient  temperature 
excursion,  electromagnetic  interference,  or  by  factors  internal  to  a 
system.  A  solid  fault  may  cause  a  transient  failure  (TR-3826) . 

Transition  count.  The  number  of  times  an  output  of  a  circuit  changes 
state. 

Transportability.  The  ability  to  convert  the  output  representation  of  an 
automatic-test-program  generator  to  a  given  automatic-test-station 
target  language.  This  is  generally  accomplished  via  a  translator. 

TRD.  An  abbreviation  for  Test  Requirements  Document. 
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PART.  Universal  asynchronous  receiver/ transmitter.  (A  hardware  category.) 

Unique  test.  A  test  which  must  be  designed  individually  and  is  generally 
executed  only  once  per  program  run. 

Unit  under  test  (UUT).  Any  system,  set,  subsystem,  assembly,  subassembly, 
and  so  forth,  undergoing  testing  (MIL-STD-1309B) . 

Unknown  state.  Most  memory  elements  used  in  sequential  circuits  are 

bistable  devices.  When  the  power  is  turned  on,  they  can  assume  either  one 
of  the  two  stable  states.  Because  they  are  normally  designed  to  have  a 
symmetrical  configuration,  the  initial  states  become  unpredictable  and 
are  called  unknown  states.  Unknown  states  can  also  be  the  result  of 
critical  races  or  oscillations. 

User  node.  A  node  is  either  an  input  or  an  output  of  a  gate  or  a  functional 
block  which  may  consist  of  many  gates.  When  a  UOT  is  described  by  a  user 
to  an  ATG  system,  nodes  are  used  to  show  the  interconnections  between 
the  gates  or  the  functional  blocks.  These  nodes  are  called  user  nodes. 
Sometimes,  a  user  is  allowed  to  define  a  macro  block  which  may  be  used 
repeatedly  in  a  UUT.  The  macro  blocks  will  be  expanded  in  terms  of  gates 
or  smaller  functional  blocks.  Thus,  the  total  number  of  nodes  will  be 
increased,  but  they  are  all  considered  as  user  nodes. 
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Wearout  failures.  Failures  during  the  period  during  which  the  failure  rate 
of  some  items  is  rapidly  increasing  due  to  deterioration  processes  (IEEE). 

Well-behaved  failure.  A  failure  whose  occurrence  produces  dependably  con¬ 
sistent  ar.d  predictable  symptoms  (TR-3826) . 
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